Managing deployment to fleets of edges using groups

Guide specification
Requirements:	None
Recommended reading:	Managing models, Model repos, Federations

Introduction

In Managing models we learned how models can be packaged into model releases and how model releases can be imported into an individual SA Engine instance. This is useful for versioning the models used on a limited amount of edges, but for federations with large amounts of connected edge devices we also need to deploy to several devices in parallel. For this purpose, edges can be divided into groups.

Publish models to servers

Each server has a default model repo (named "default"). To publish a model to a server repository you use the models:publish command. The following query publishes version 1.0 of model "my-model" to the server (how you create versioned releases of a model is described in Managing models):

models:publish("my-model", "1.0");

This uploads the model release to the default repository on the server. You can specify the repository by providing the repository name as the first parameter.

models:publish("default", "my-model", "1.0");

To manually load a published model release on an edge device, use the models:import command. The command should be executed on the edge. This is useful for testing and for managing small federations, but not for managing fleets of edges.

//peer: my-edge
models:import("my-model", "1.0");

Deploy a model directly to an edge

You can deploy a model in development directly to one or more edge devices with models:deploy. For example, the following query deploys the model "my-model" to the edge "my-edge".

models:deploy("my-edge", "my-model")

The model is deployed from the local model folder ($SA_HOME/models). Both the local machine and the edge device must be connected to the same federation.

You can also deploy a model directly to multiple edge devices at the same time by providing a vector of edge names. For example, the following query deploys the model "my-model" to the edges "my-edge-1" and "my-edge-2":

models:deploy(["my-edge-1", "my-edge-2"], "my-model")

When a model is deployed to an edge, the OSQL code in the model will be available on the edge device disk, as well as loaded into memory. If SA Engine on the edge is restarted, the model can be loaded into memory again using the models:load command. The command should be executed on the edge.

//peer: my-edge
models:load("my-model");

Import and load reloads the model

models:import loads a published and released version from the server.

models:deploy sends and loads a version in development to a defined edge.

models:load loads a local version that has previously been deployed.

Even though these models come from different sources, they replace each other. It is the last loaded version of the model that is used, regardless of which code is newer.

When sa.engine loads a model, all functions belonging to the previously loaded version will first be dropped. After that, the functions belonging to the currently loaded version will be defined. It is not advisable to use the functions during this process.

Edge groups

Experimental

The edge group functionality is in experimental state and is subject to change

An edge group is a named reference between edges and models. When an edge is connected to a federation, it will regularly poll the nameserver and ask which models of what version it should be running.

The server will then check which edge group the edge belongs to, and which models are in the group. If the models, or the desired version of the models, has changed, the new release will be automatically deployed to the edge.

It is recommended to always use edge groups to manage desired models of edges. To set a model to only one edge, create an edge group with only that edge in it.

Model groups are managed on the Nameserver

Since it is the nameserver that contains the list of model groups, all queries that manage model groups must be executed on the nameserver.

This can be done by either prefixing the query

//peer: nameserver
<query>

Or if you use VSCode or SA Studio you can select the nameserver as your default peer for executing queries. Information on how to do this can be found here for VSCode and here for SA Studio.

Managing edges in edge groups

You add an edge to a group by calling edge:add_group. Groups are automatically created when an edge is added to it. For example, the following query will add an edge named "MY-EDGE-1" to the edge group "staging-area-edges":

//peer: nameserver
edge:add_group("MY-EDGE-1", "staging-area-edges");

If "staging-area-edges" does not exist it will be created.

Edges can belong to several groups. To view which groups a specific edge belongs to you call edge:groups.

//peer: nameserver
edge:groups("MY-EDGE-1");

This will return a vector with all model groups the edge is assigned to, in this case [staging-area-edges].

To remove the edge from a group, use edge:remove_group. For example, the following query will remove the edge named "MY-EDGE-1" from the edge group "staging-area-edges". It will not remove the edge group.

//peer: nameserver
edge:remove_group("MY-EDGE-1", "staging-area-edges");

The all group

There is a pre-defined group called all, which can always be used set a model to all the edges in a federation.

Assigning models to edge groups

Models need to be available in the model repository before they are added to a group. Run models:installed() on the nameserver to get a list of all available models. Read more about Model repos here.

To add a model to a group use the command edge:set_group_models.

For example, the following query adds the model "my-sa-model" version 5.3 in the default repository to the group "staging-area-edges":

//peer: nameserver
edge:set_group_models("staging-area-edges",
                      [{"repo": "default",
                        "name": "my-sa-model",
                        "version": "5.3"}]);

It is possible to add several different models to a group, but only one version of each model. If, for example, a version 5.4 of "my-sa-model" is assigned to "staging-area-edges", it will replace 5.3. Downgrading is also supported, it is the latest assigned version that is used, not the highest version number.

Models in an Edge group replace any manually deployed models

The model version that is automatically deployed to the edges in a group will replace any manually loaded version of the same model in those edges. If a manual deployment is done after setting up automatic deployment, it will temporarily load, but it will be replaced again as soon as the edge polls the server.

Edge state

All edges have a specific state. The state includes the edge mode (INTERACTIVE, ONLINE, or AUTONOMOUS). It also shows which models are loaded and the last load errors.

To get the state of an edge, call

edge:actual_state("EDGE_NAME");

USE UPPER CASE

Note that the edge name parameter in edge:actual_state() must be UPPER CASE.

This will return a record that specifies the edge mode, loaded models and load errors, for example

{"models:loaded":['my-sa-model@5.3'],"last_load_errors":[],"edge_mode":'INTERACTIVE'}

Edge modes

Edges in a federation are in one of three possible modes - INTERACTIVE, ONLINE, or AUTONOMOUS. The modes indicate to what extent the edges interacts with the server.

Mode	Interactivity	Edge queries	Creation
INTERACTIVE	Have an uplink to the server. Edge state can be set from the server, which will trigger a poll from the edge.	Possible to send continous queries from the server and see the results.	Default mode for edges.
ONLINE	Has no uplink, but polls the server at regular intervals. Edge state can be set from the server, the request will await next poll.	Runs models and queries independently. To store the results and send to the server when connecting, use Store and Forward.	Set an edge to online explicitly.
AUTONOMOUS	Not reachable from the server at all, managed individually.	No, only runs models started at the edge.	Start the SA Engine instance at the edge in autonomous mode.

It is convenient to use interactive edges while developing models. For large fleets in production, it is advisable to use online mode to limit the load on the server.

To set the mode of an edge, use edge:set_mode, for example this query will set the mode of "edge_name" to ONLINE.

edge:set_mode("edge_name", "ONLINE");

Loaded models

The second part of the edge state is the loaded models. It shows the models that are loaded into the SA Engine memory, and is currently running on that edge.

After setting a model to an edge group with set_group_models, it will take a while before it completes loading into memory, dependinging on edge mode and poll frequency. To view the desired state - which model the edge will be told to run on next poll - use edge:get_config:

// nameserver
edge:get_config("edge_name");