# Managing models

Studio guide (Requires signup)

This is an interactive guide using Studio code blocks. To learn more about what this means you can read the Interactive documentation! section.

A few things to think about when using Studio guides:

1. 📋 Execute code blocks in order: Code blocks in interactive guides are often designed to be executed in the order they appear. So if you skip a codeblock subsequent codeblocks might fail due to missing dependencies.
2. 📝 Edit examples: You can edit the code in the code blocks if you you want to try changing the example.
3. 📈 Change Visualization: You can change how the output of a query is visualized by specifying //plot: <vis-format> at the top of the code block. Available visualization formats are described in the Visualization chapter of the SA Studio manual.
4. 🤖 Run code on edge devices: You can run code on an edge device connected to your SA Studio instance by specifying //peer: <edge-id> at the top of the code block.
Guide specification
Guide type:Studio guide
Requirements:None

## Introduction​

In this guide we look at how model releases work in SA Engine. You should read the Data and models section in the System Tutorial first to fully understand what models are and how they work.

## What is a model​

A minimal model in SA Engine is a folder with a master.osql file in it. You load models with the load_model() function. When you load a model, the following two things happen:

1. SA Engine changes the current working directory to the model folder.
2. The master.osql file is loaded (this means that SA Engine runs all instructions in master.osql).

You can add any number of files (like OSQL files and data files) and directories to the model folder.

Below is a sample directory tree from SA_HOME for two models model1 and model2

SA/└─ models/   ├─ model1/   │  └─ master.osql   └─ model2/      └─ master.osql

### File paths inside models​

In master.osql you can use the function file:current_folder() to make relative file paths to files in your model. For example, if you want to load a data file mydata.csv in the model folder you can specify the path as file:current_folder() + mydata.csv. Generally the function file:current_folder() points to the current working directory of the SA Engine process. But if used in an OSQL file that is being loaded, then it refers to the folder of the file being loaded.

## What are the different components of the model ecosystem​

There are three physical locations where a model can reside:

1. User models (SA_HOME/models) - This is your user model folder. Here your models are organized as folders containing osql files, as we saw in the What is a model section. Here is where you can edit your model and from here a model can be loaded into SA Engine. You can use the function user_models() to list all models in your user models directory.

2. Local model repository (SA_HOME/model_releases) - This is your local folder for model packages. A model package is a .s.fcz file which can be imported into SA Engine. Models can be stored here and loaded into SA Engine from here. You can use the function models:installed_locally() to list all models in your local model repo.

3. SA Engine instance - Models can be loaded (or imported) into an SA Engine instance. If a model has been loaded (or imported) into SA Engine, then the model file master.osql has been executed and all tables and functions in master.osql will be available in the SA Engine instance. The model (or rather its contents) then resides in the database image of the SA Engine instance. You can use the function loaded_models() to list all the user models that have been loaded into the SA Engine.

The following figure shows a visual overview of the model ecosystem along with the functions used to manage the models.

## Creating a model​

You create a model with the models:create() function. It will create a new model in your User models.

Let's create a new model:

models:create("my_model");
Not connected

To run this code block you must be logged in and your studio instance must be started.

We can verify that the model was created by listing the models in our User models:

user_models();
Not connected

To run this code block you must be logged in and your studio instance must be started.

The result should include our new model:

"my_model"

## Releasing (packaging) a model​

When releasing a model, SA Engine always packs the model into a SAPACK (.s.fcz) file format.

SAPACK file format

SAPACK is a file format where all files are first compressed using Lz77 and then concatenated together with a header telling the unpacker what compression to use. Essentially it is a light-weight version of tar + gzip.

You can release a model with the models:create_release() function. Releasing a model takes the model from your User models, packs it and stores the compressed model file in your Local model repository.

Try creating a release of our new model:

models:create_release("my_model","1.0");
Not connected

To run this code block you must be logged in and your studio instance must be started.

We can verify that a release was created by running the function models:installed_locally(), which lists the models in your Local model repository:

models:installed_locally();
Not connected

To run this code block you must be logged in and your studio instance must be started.

The result should include the model we just released:

"my_model@1.0.s.fcz"

Released models have the format <name>@<version>.s.fcz.

## Importing a model​

Importing a model will unpack the model from the Local model repository into your User models and load it into your SA Engine instance.

You import a model with the models:import() function. Note that the model must exist in your Local model repository:

models:import("my_model","1.0");
Not connected

To run this code block you must be logged in and your studio instance must be started.

We can check that the model was imported with the loaded_models() function:

loaded_models();
Not connected

To run this code block you must be logged in and your studio instance must be started.

The result should include the model we just loaded:

"my_model@1.0"

If we create a new version of the model and reload it, it will be replaced:

models:create_release("my_model","1.1");models:import("my_model","1.1");loaded_models();
Not connected

To run this code block you must be logged in and your studio instance must be started.

The version of my_model in the result from loaded_models() should now be 1.1:

"my_model@1.1"

## Installing a model​

You can use the models:install() function to install a packaged model from an s.fcz file. It will install it into your Local model repository and tag it with a new name and version.

Try installing a .s.fcz file into your Local model repository and give it a name and version:

models:install(sa_home()+"model_releases/my_model@1.0.s.fcz",               "my_model2","1.1");
Not connected

To run this code block you must be logged in and your studio instance must be started.

We can verified that the new model was installed with the models:installed_locally() function:

models:installed_locally();
Not connected

To run this code block you must be logged in and your studio instance must be started.

The result should include the model we just installed:

"my_model2@1.1.s.fcz"
Tip

With this method you can load any .s.fcz file from disk.

## Model repository​

note

For the examples in this section to work you need to have an edge with the name 'edge1' connected to your SA Studio instance. How to connect edge devices to SA Studio is described in the Connecting edge devices chapter of the SA Studio manual.

A model repository is a service where SA Engines can access models by name and version via HTTP/REST calls. If no repository is configured an SA Engine will use the Nameserver of the federation as a repository.

For instance, if we import my_model on the edge it will fetch the .s.fcz file from the Nameserver and install it.

First we verify that my_model is not already installed on the edge:

//peer: edge1models:installed_locally();
Not connected

To run this code block you must be logged in and your studio instance must be started.

The result should not include any version of my_model.

The we import the model:

//peer: edge1models:import("my_model","1.0");
Not connected

To run this code block you must be logged in and your studio instance must be started.

Now we verify that the model has been installed on the edge device:

//peer: edge1models:installed_locally();
Not connected

To run this code block you must be logged in and your studio instance must be started.

The result should include the installed model:

"my_model@1.0"

This shows that you just installed a model to an edge device using a central model repository!

The next section will show you how to set up custom repositories.