Managing models in a federation

Guide specification
Requirements:	None
Recommended reading:	Managing models locally

In Model management, we have seen how models can be managed locally. When working in a federation, models can reside in the client as well as on the edge or on the nameserver. This guide will walk through the recommended way of working with models in federations.

Model locations

Consider the federation below. A nameserver is running on a server or in the cloud. An analyst developing models on a laptop is using a local client that is connected to the nameserver. There is also an edge instance connected to the federation, which is where the model will ultimately be used.

A small federation with a nameserver in the cloud, a client on a local computer and an edge.

Just like when working locally, there are three physical locations where models can reside in the client;

1. User models (The SA_HOME/models folder)

2. Local model repository (The SA_HOME/model_releases folder)

3. SA Engine instance - Models can be loaded (or imported) into the local SA Engine instance. This is useful to run the functions in the model locally.

When working with federations, released models are also sent to a central model repository where they are available to edges. As default, the nameserver is used as the central model repository.

On the edge device, models normally reside in the edge SA Engine instance, where the functions are used to analyse live signals. The edge device also has an SA_HOME folder where models can be stored locally.

The SA Home folder

All SA Engine instances have an SA_HOME folder associated to them. This is always a path to a folder on the same machine as where the SA Engine instance is running. Often similar paths are used on eg. the client and the edges, but the physical location is different.

The SA_HOME path can be set as an environment variable on the machine. It can also be set explicitly:

• When starting from the command line by using the -f flag, for example sa.engine -f my\path\SA
• When using VS Code the clients path can be configured in the VS Code startup file

Sending models to the edge

There are two ways of sending models from the client to the edge. The user models in the client models folder can be deployed directly to the edge. The released models in the local model repository need to be published to the central model repository, then they can be imported on the edge instance.

When developing a model, it is common to use deploy to test out changes on the edge, while publish-import is used to handle released models for testing or operation.

Model ecosystem in a small federation. Models in development can be deployed while released models are published and imported.

Deploy a model directly to an edge

You can deploy a model in development directly to one or more edge devices with models:deploy. For example, the following query deploys the model "my-model" to the edge "my-edge".

models:deploy("my-edge", "my-model")

The model is deployed from the local model folder ($SA_HOME/models). Both the local machine and the edge device must be connected to the same federation.

You can also deploy a model directly to multiple edge devices at the same time by providing a vector of edge names. For example, the following query deploys the model "my-model" to the edges "my-edge-1" and "my-edge-2":

models:deploy(["my-edge-1", "my-edge-2"], "my-model")

Publish models to the server

To publish a model to a server repository you use the models:publish command. The following query publishes version 1.0 of model "my-model" to the server:

models:publish("my-model", "1.0");

This uploads the model release to the default repository on the server. You can specify the repository by providing the repository name as the first parameter.

models:publish("default", "my-model", "1.0");

Import published models

To manually load a published model release on an edge device, use the models:import command. The command should be executed on the edge.

//peer: my-edge
models:import("my-model", "1.0");

Note

There can only be one model with the same name in the edge SA Engine instance. Even though imported and deployed models come from different sources, they replace each other. It is the last loaded version of the model that is used, regardless of which code is newer.

When sa.engine loads a model, all functions belonging to the previously loaded version will first be dropped. After that, the functions belonging to the currently loaded version will be defined. It is not advisable to use the functions during this process.

Model storage on the edge

The edge device also has an SA_HOME folder where models are stored locally. When models are deployed or imported on the edge, the OSQL code is first sent to the device, where it is stored in the SA_HOME folder, and then loaded into the instance. When deploying code in development directly, it is stored in the user model folder on the edge. When importing a published model, it is stored in the local model repository on the edge.

Model ecosystem in a small federation. When models are deployed they are also stored in the user model folder on the edge. When models are imported they are also stored in the local model repository on the edge.

Reloading previously deployed model

If SA Engine on the edge is restarted, a previously deployed model can be loaded into memory again using the models:load command. The command should be executed on the edge.

//peer: my-edge
models:load("my-model");

Similarily, a previously imported model can be imported again using the same models:import command.

Sending models in local federations

A Local federation can be used for testing models on the edge. A nameserver and a client are started on the developer machine and the edge is connected over the local network. In this case, the nameserver and client instances of SA Engine share the same SA_HOME folder and model repository. There is no need to publish model releases from client to server, the released models can be imported directly.

Model ecosystem in a local federation. Since the client and the nameserver run on the same machine, they share the same local model repository.