Managing deployment to fleets of edges using groups

Guide specification
Requirements:	None
Recommended reading:	Managing models in a federation

Introduction

In Managing models in a federation we learned how models can be deployed to edges and how model releases can be published to a central model repository and imported into an individual edge SA Engine instance.

This is useful for versioning the models used on a limited amount of edges, but for federations with large amounts of connected edge devices we also need to deploy to several devices in parallel. For this purpose, edges can be divided into groups that automatically get model releases from the central model repository.

Edge groups

Experimental

The edge group functionality is in experimental state and is subject to change

An edge group is a named reference between edges and models. When an edge is connected to a federation, it will regularly poll the nameserver and ask which models of what version it should be running.

The server will then check which edge group the edge belongs to, and which models are in the group. If the models, or the desired version of the models, has changed, the new release will be automatically deployed to the edge.

It is recommended to always use edge groups to manage desired models of edges. To set a model to only one edge, create an edge group with only that edge in it.

Model groups are managed on the Nameserver

Since it is the nameserver that contains the list of model groups, all queries that manage model groups must be executed on the nameserver.

This can be done by either prefixing the query

//peer: nameserver
<query>

Or if you use VSCode or SA Studio you can select the nameserver as your default peer for executing queries. Information on how to do this can be found here for VSCode and here for SA Studio.

Managing edges in edge groups

You add an edge to a group by calling edge:add_group. Groups are automatically created when an edge is added to it. For example, the following query will add an edge named "MY-EDGE-1" to the edge group "staging-area-edges":

//peer: nameserver
edge:add_group("MY-EDGE-1", "staging-area-edges");

If "staging-area-edges" does not exist it will be created.

Edges can belong to several groups. To view which groups a specific edge belongs to you call edge:groups.

//peer: nameserver
edge:groups("MY-EDGE-1");

This will return a vector with all model groups the edge is assigned to, in this case [staging-area-edges].

To remove the edge from a group, use edge:remove_group. For example, the following query will remove the edge named "MY-EDGE-1" from the edge group "staging-area-edges". It will not remove the edge group.

//peer: nameserver
edge:remove_group("MY-EDGE-1", "staging-area-edges");

The all group

There is a pre-defined group called all, which can always be used set a model to all the edges in a federation.

Assigning models to edge groups

Models need to be available in the model repository before they are added to a group. Run models:installed() on the nameserver to get a list of all available models.

To add a model to a group use the command edge:set_group_models.

For example, the following query adds the model "my-sa-model" version 5.3 in the default repository to the group "staging-area-edges":

//peer: nameserver
edge:set_group_models("staging-area-edges",
                      [{"repo": "default",
                        "name": "my-sa-model",
                        "version": "5.3"}]);

It is possible to add several different models to a group, but only one version of each model. If, for example, a version 5.4 of "my-sa-model" is assigned to "staging-area-edges", it will replace 5.3. Downgrading is also supported, it is the latest assigned version that is used, not the highest version number.

Models in an Edge group replace any manually deployed models

The model version that is automatically deployed to the edges in a group will replace any manually loaded version of the same model in those edges. If a manual deployment is done after setting up automatic deployment, it will temporarily load, but it will be replaced again as soon as the edge polls the server.

Edge state

All edges have a specific state. The state includes the edge mode (INTERACTIVE, ONLINE, or AUTONOMOUS). It also shows which models are loaded and the last load errors.

To get the state of an edge, call

edge:actual_state("EDGE_NAME");

This will return a record that specifies the edge mode, loaded models and load errors, for example

{"models:loaded":['my-sa-model@5.3'],"last_load_errors":[],"edge_mode":'INTERACTIVE'}

Loaded models

The vector in models:loaded shows the models that are loaded into the SA Engine memory, and is currently running on that edge.

After setting a model to an edge group with set_group_models, it will take a while before it completes loading into memory, dependinging on edge mode and poll frequency. To view the desired state - which model the edge will be told to run on next poll - use edge:get_config:

// nameserver
edge:get_config("edge_name");