Managing models locally
Guide specification | |
---|---|
Guide type: | Studio code |
Requirements: | None |
Recommended reading: | Data and models |
Introduction
In this guide we look at how model releases work in SA Engine. You should read the Data and models section in the System Tutorial first to fully understand what models are and how they work.
What is a model
A minimal model in SA Engine is a folder with a master.osql
file in it. You load models with the models:load()
function. When you load a model, the following two things happen:
- SA Engine changes the current working directory to the model folder.
- The
master.osql
file is loaded (this means that SA Engine runs all instructions inmaster.osql
).
You can add any number of files (like OSQL files and data files) and directories to the model folder.
Below is a sample directory tree from SA_HOME for two models model1
and model2
SA/
└─ models/
├─ model1/
│ └─ master.osql
└─ model2/
└─ master.osql
File paths inside models
In master.osql
you can use the function file:current_folder()
to make relative file paths to files in your model. For example, if you want to load a data file mydata.csv
in the model folder you can specify the path as file:current_folder() + mydata.csv
. Generally the function file:current_folder()
points to the current working directory of the SA Engine process. But if used in an OSQL file that is being loaded, then it refers to the folder of the file being loaded.
What are the different components of the model ecosystem
There are three physical locations where a model can reside:
User models (
SA_HOME/models
) - This is your user model folder. Here your models are organized as folders containing osql files, as we saw in the What is a model section. Here is where you can edit your model and from here a model can be loaded into SA Engine. You can use the functionmodels:list()
to list all models in your user models directory.Local model repository (
SA_HOME/model_releases
) - This is your local folder for model packages. A model package is a.s.fcz
file which can be imported into SA Engine. Models can be stored here and loaded into SA Engine from here. You can use the functionmodels:installed()
to list all models in your local model repo.SA Engine instance - Models can be loaded (or imported) into an SA Engine instance. If a model has been loaded (or imported) into SA Engine, then the model file
master.osql
has been executed and all tables and functions inmaster.osql
will be available in the SA Engine instance. The model (or rather its contents) then resides in the database image of the SA Engine instance. You can use the functionmodels:loaded()
to list all the user models that have been loaded into the SA Engine.
The following figure shows a visual overview of the model ecosystem along with the functions used to manage the models.
An overview of the SA Engine model ecosystem.
Creating a model
Models can be created and loaded using the graphical interfaces in VS Code or SA Studio.
You can also create a model with the models:create()
function. It will create a new model in your User models.
Let's create a new model:
models:create("my_model");
To run this code block you must be logged in and your studio instance must be started.
We can verify that the model was created by listing the models in our User models:
models:list();
To run this code block you must be logged in and your studio instance must be started.
The result should include our new model:
"my_model"
Releasing (packaging) a model
When releasing a model, SA Engine always packs the model into a SAPACK (.s.fcz) file format.
SAPACK is a file format where all files are first compressed using Lz77 and then concatenated together with a header telling the unpacker what compression to use. Essentially it is a light-weight version of tar + gzip.
You can release a model with the models:create_release()
function. Releasing a model takes the model from your User models, packs it and stores the compressed model file in your Local model repository.
Try creating a release of our new model:
models:create_release("my_model","1.0");
To run this code block you must be logged in and your studio instance must be started.
We can verify that a release was created by running the function models:installed()
, which lists the models in your Local model repository:
models:installed();
To run this code block you must be logged in and your studio instance must be started.
The result should include the model we just released:
"my_model@1.0.s.fcz"
Released models have the format <name>@<version>.s.fcz
.
Importing a model
Importing a model will unpack the model from the Local model repository into your User models and load it into your SA Engine instance.
You import a model with the models:import()
function. Note that the model must exist in your Local model repository:
models:import("my_model","1.0");
To run this code block you must be logged in and your studio instance must be started.
We can check that the model was imported with the models:loaded()
function:
models:loaded();
To run this code block you must be logged in and your studio instance must be started.
The result should include the model we just loaded:
"my_model@1.0"
If we create a new version of the model and reload it, it will be replaced:
models:create_release("my_model","1.1");
models:import("my_model","1.1");
models:loaded();
To run this code block you must be logged in and your studio instance must be started.
The version of my_model
in the result from models:loaded()
should now be 1.1:
"my_model@1.1"
Installing a model
You can use the models:install()
function to install a packaged model from an s.fcz
file. It will install it into your Local model repository and tag it with a new name and version.
Try installing a .s.fcz
file into your Local model repository and give it a name and version:
models:install(sa_home()+"model_releases/my_model@1.0.s.fcz",
"my_model2","1.1");
To run this code block you must be logged in and your studio instance must be started.
We can verified that the new model was installed with the models:installed()
function:
models:installed();
To run this code block you must be logged in and your studio instance must be started.
The result should include the model we just installed:
"my_model2@1.1.s.fcz"
With this method you can load any .s.fcz
file from disk.