Managing models

Guide specification
Guide type:	Studio code
Requirements:	None
Recommended reading:	Data and models

Introduction

In this guide we look at how model releases work in SA Engine. You should read the Data and models section in the System Tutorial first to fully understand what models are and how they work.

What is a model

A minimal model in SA Engine is a folder with a master.osql file in it. You load models with the models:load() function. When you load a model, the following two things happen:

1. SA Engine changes the current working directory to the model folder.
2. The master.osql file is loaded (this means that SA Engine runs all instructions in master.osql).

You can add any number of files (like OSQL files and data files) and directories to the model folder.

Below is a sample directory tree from SA_HOME for two models model1 and model2

SA/
└─ models/
   ├─ model1/
   │  └─ master.osql
   └─ model2/
      └─ master.osql

File paths inside models

In master.osql you can use the function file:current_folder() to make relative file paths to files in your model. For example, if you want to load a data file mydata.csv in the model folder you can specify the path as file:current_folder() + mydata.csv. Generally the function file:current_folder() points to the current working directory of the SA Engine process. But if used in an OSQL file that is being loaded, then it refers to the folder of the file being loaded.

What are the different components of the model ecosystem

There are three physical locations where a model can reside:

1. User models (SA_HOME/models) - This is your user model folder. Here your models are organized as folders containing osql files, as we saw in the What is a model section. Here is where you can edit your model and from here a model can be loaded into SA Engine. You can use the function models:list() to list all models in your user models directory.

2. Local model repository (SA_HOME/model_releases) - This is your local folder for model packages. A model package is a .s.fcz file which can be imported into SA Engine. Models can be stored here and loaded into SA Engine from here. You can use the function models:installed() to list all models in your local model repo.

3. SA Engine instance - Models can be loaded (or imported) into an SA Engine instance. If a model has been loaded (or imported) into SA Engine, then the model file master.osql has been executed and all tables and functions in master.osql will be available in the SA Engine instance. The model (or rather its contents) then resides in the database image of the SA Engine instance. You can use the function models:loaded() to list all the user models that have been loaded into the SA Engine.

The following figure shows a visual overview of the model ecosystem along with the functions used to manage the models.

An overview of the SA Engine model ecosystem.

Creating a model

You create a model with the models:create() function. It will create a new model in your User models.

Let's create a new model:

models:create("my_model");

Not connected

To run this code block you must be logged in and your studio instance must be started.

We can verify that the model was created by listing the models in our User models:

models:list();

Not connected

To run this code block you must be logged in and your studio instance must be started.

The result should include our new model:

"my_model"

Releasing (packaging) a model

When releasing a model, SA Engine always packs the model into a SAPACK (.s.fcz) file format.

SAPACK file format

SAPACK is a file format where all files are first compressed using Lz77 and then concatenated together with a header telling the unpacker what compression to use. Essentially it is a light-weight version of tar + gzip.

You can release a model with the models:create_release() function. Releasing a model takes the model from your User models, packs it and stores the compressed model file in your Local model repository.

Try creating a release of our new model:

models:create_release("my_model","1.0");

Not connected

To run this code block you must be logged in and your studio instance must be started.

We can verify that a release was created by running the function models:installed(), which lists the models in your Local model repository:

models:installed();

Not connected

To run this code block you must be logged in and your studio instance must be started.

The result should include the model we just released:

"my_model@1.0.s.fcz"

Released models have the format <name>@<version>.s.fcz.

Importing a model

Importing a model will unpack the model from the Local model repository into your User models and load it into your SA Engine instance.

You import a model with the models:import() function. Note that the model must exist in your Local model repository:

models:import("my_model","1.0");

Not connected

To run this code block you must be logged in and your studio instance must be started.

We can check that the model was imported with the models:loaded() function:

models:loaded();

Not connected

To run this code block you must be logged in and your studio instance must be started.

The result should include the model we just loaded:

"my_model@1.0"

If we create a new version of the model and reload it, it will be replaced:

models:create_release("my_model","1.1");
models:import("my_model","1.1");
models:loaded();

Not connected

To run this code block you must be logged in and your studio instance must be started.

The version of my_model in the result from models:loaded() should now be 1.1:

"my_model@1.1"

Installing a model

You can use the models:install() function to install a packaged model from an s.fcz file. It will install it into your Local model repository and tag it with a new name and version.

Try installing a .s.fcz file into your Local model repository and give it a name and version:

models:install(sa_home()+"model_releases/my_model@1.0.s.fcz",
               "my_model2","1.1");

Not connected

To run this code block you must be logged in and your studio instance must be started.

We can verified that the new model was installed with the models:installed() function:

models:installed();

Not connected

To run this code block you must be logged in and your studio instance must be started.

The result should include the model we just installed:

"my_model2@1.1.s.fcz"

Tip

With this method you can load any .s.fcz file from disk.

Model repository

note

For the examples in this section to work you need to have an edge with the name 'edge1' connected to your SA Studio instance. How to connect edge devices to SA Studio is described in the Connecting edge devices chapter of the SA Studio manual.

A model repository is a service where SA Engines can access models by name and version via HTTP/REST calls. If no repository is configured an SA Engine will use the Nameserver of the federation as a repository.

For instance, if we import my_model on the edge it will fetch the .s.fcz file from the Nameserver and install it.

First we verify that my_model is not already installed on the edge:

//peer: edge1
models:installed();

Not connected

To run this code block you must be logged in and your studio instance must be started.

The result should not include any version of my_model.

The we import the model:

//peer: edge1
models:import("my_model","1.0");

Not connected

To run this code block you must be logged in and your studio instance must be started.

Now we verify that the model has been installed on the edge device:

//peer: edge1
models:installed();

Not connected

To run this code block you must be logged in and your studio instance must be started.

The result should include the installed model:

"my_model@1.0"

This shows that you just installed a model to an edge device using a central model repository!

---

The next section will show you how to set up custom repositories.