Fuel efficiency

Note

Use this guide while running the SA/models/fuel_efficiency model in the public github repository sa.example-models. The model requires SA Engine version 6.0.3 or later.

This model predicts fuel efficency of automobiles from the Auto-MPG dataset using trained regression models. The ML models are trained using Python and Tensorflow, and inferred using OSQL.

Fuel prediction using regression and the Auto MPG dataset is described in this Tensorflow tutorial. Refer to this tutorial for a walk-through of the contained ML concepts.

1. Inference with OSQL

The system model NN contain functions for inferring neural networks. Load it to use the functions:

system_models:load("nn");

Functions such as dense and conv1d make up the layers. The functions take input data and weights as arguments. To stack the layers, the functional approach is used and output from one layer is passed as input to the next one.

To create a model, we can write a function that contains the layers and takes all of their weight arrays as input. The dimension of the layers is given by the dimension of the weight arrays.

In inference.osql in the fuel efficiency model, the first example of such a model is a linear regression model. It has a normalization layer, and a dense layer:

create function linear_regression(
    Array of f32 norm_weights, Array of f32 norm_bias, 
    Array of f32 dense_weights, Array of f32 dense_bias, 
    Array of f32 data) -> Array of f32
/* Inference of linear regression with normalization, using given weights */
as select out 
    from Array of f32 out, Array of f32 normalized
    where normalized = normalize(data, norm_weights, norm_bias)
    and out = dense(normalized, dense_weights, dense_bias, #'identity');

The same function can be used for single or multi variable regression, depending on the form of the weight array.

To test this as a linear transformation, y = mx + b, we can use mean 0 and variance 1 for the normalization layer, and thus leaving the normalized output equal to the input. With the data x=2, the weight m=1.2 and the bias b=5, y = 2*1.2+5 = 7.4. This test is found in inference.test.osql:

validate "linear regression"
check linear_regression(array("f32", [0]), array("f32", [1]),
                        array("f32", [[1.2]]), array("f32", [5]),
                        array("f32", [2])) => array("f32", [7.4]);

In the linear regression above, no activation function is wanted, so the identity function is used.

Activation functions such as relu and softmax, are also loaded with the NN system model. In inference.osql, the relu activation is used in a model called my_dnn, with a normalization layer and three dense layers. The activation function is applied to the output of each layer by nesting the functions:

create function my_dnn(
    Array of f32 norm_weights, Array of f32 norm_bias, 
    Array of f32 dense_weights, Array of f32 dense_bias,
    Array of f32 dense_1_weights, Array of f32 dense_1_bias,
    Array of f32 dense_2_weights, Array of f32 dense_2_bias, 
    Array of f32 data) -> Array of f32
/* A deep neural network with 3 dense layer and normalization */
as select out 
    from Array of f32 out, Array of f32 normalized, 
    Array of f32 densed, Array of f32 densed_1
    where normalized = normalize(data, norm_weights, norm_bias)
    and densed = relu(dense(normalized, dense_weights, dense_bias, #'identity'))
    and densed_1 = relu(dense(densed, dense_1_weights, dense_1_bias, #'identity'))
    and out = dense(densed_1, dense_2_weights, dense_2_bias, #'identity');

As we can see, when the model grows larger, there is a need to store the weights rather than writing them each time the model is called.

2. Weights in OSQL

2.1 Storing and using weights

SQL tables are the recommended way of storing metadata of models, such as the weights of trained models. In weights.osql a table is defined that will contain the weights of several models trained in fuel_efficiency.

create table Weight(model Charstring, 
                    layer Charstring, 
                    value Array of F32, 
                    unique(model, layer));

Weights are added with the SQL insert statement. We add a convenience function for adding:

create function w(charstring m, charstring l, array of f32 v) -> integer
as insert into Weight values(m, l, v);

and another for accesseing weights by model and layer names:

create function w(charstring m, charstring l) -> Bag of Array of f32
as select w.value from Weight w where w.model = m and w.layer = l;

For example, lets store weights for the linear regression model used above:

w("Line", "normalization", array("F32",[0]));
w("Line", "normalization_bias", array("F32",[1]));
w("Line", "dense", array("F32",[[1.2]]));
w("Line", "dense_bias", array("F32",[5]));

Now we can call the linear model using the stored weights:

linear_regression(w("Line", "normalization"), w("Line", "normalization_bias"),
                  w("Line", "dense"), w("Line", "dense_bias"),
                  array("f32", [2]));

Another setup could be to access the weights inside the model function:

create function linear_regression_Line(Array of f32 data) -> Array of f32
/* Inference of linear regression with normalization, using Line weights */
as select out 
    from Array of f32 out, Array of f32 normalized
    where normalized = normalize(data, w("Line", "normalization"), w("Line", "normalization_bias"))
    and out = dense(normalized, w("Line", "dense"), w("Line", "dense_bias"), #'identity');

and call it with just the data:

linear_regression_Line(array("f32", [2]));

2.2 Exporting weights to file

When weights are stored in an SQL table, they are saved in the SA Engine local in-memory database. If we also make the weights part of the SA model, they are included when the SA model is deployed to edges. Since the inference uses OSQL only, this means that we can infer on any edge device.

To make the weights a part of the fuel_efficiency model, we will export them to a CSV file. In weights.osql, an export function that takes the trained model name as input is defined.

It stores the CSV file inside the model, but here we will just put it in sa_home():

create function export_weights(Charstring name) -> Boolean
as csv:write_file(sa_home() + name + '.csv',
(select [w.layer, json:stringify(w.value)] from Weight w where w.model = name));

To use the weights in another instance of SA Engine (such as an edge), the exported file need to be imported into that instance's in-memory database. For this, an import function is defined:

create function import_weights(charstring model_name) -> object
as select w(model_name, layer, cast(json:unstringify(value) as array of f32)) 
from Charstring layer, Charstring value
where [layer,value] in csv:file_stream(sa_home() + model_name +'.csv');

To try the import function on a larger example, download a file with weights for the DNN model:

http:download_file("https://assets.streamanalyze.com/docs/guides/example-models/DNN.csv",{}, sa_home() + "DNN.csv");

Import the downloaded model into SQL:

import_weights("DNN");

We can use the imported weights to infer the DNN model:

my_dnn(
    w("DNN", "normalization"), w("DNN", "normalization_bias"), 
    w("DNN", "dense"), w("DNN", "dense_bias"), 
    w("DNN", "dense_1"), w("DNN", "dense_1_bias"), 
    w("DNN", "dense_2"), w("DNN", "dense_2_bias"),  
    ARRAY('f32', [8, 390, 190, 3850, 8.5, 70, 0, 0, 1])
);

3. Training the model with Python Tensorflow

In this chapter we will process data and train regression models using Python and Tensorflow. The functions can also be called from OSQL, using foreign functions and the SA Engine Python Interface. This will enable us to extract the weights from the trained models and use them in the OSQL models.

3.1 Setting up the python environment

OSQL can call python functions and use any python package, as long as it is running in an environment where the package is available.

The python code in this model uses tensorflow 2.13.0 or later, numpy 1.24.3 or later and pandas. To ensure that the correct environment is running, it is recommended to start the SA Engine client, for example VS Code, from within it.

To set up an environment using anaconda on Windows, the following commands can be run from the root folder of sa.example-models:

cd anaconda
conda env create --file environment.yml
cd ..
conda activate sa-nn
pip install tensorflow==2.13.0
code .

3.2 Prepare and import the dataset

The data is downloaded and pre-processed using the python pandas functions described in the tensorflow tutorial.

In dataset.py:

import pandas as pd
import numpy as np

# Download and import dataset
column_names = ['MPG', 'Cylinders', 'Displacement', 'Horsepower', 'Weight',
                'Acceleration', 'Model Year', 'Origin']
url = 'http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data'
raw_dataset = pd.read_csv(url, names=column_names,
                          na_values='?', comment='\t',
                          sep=' ', skipinitialspace=True)
dataset = raw_dataset.copy()

# Clean the dataset
dataset = dataset.dropna()
dataset['Origin'] = dataset['Origin'].map({1: 'USA', 2: 'Europe', 3: 'Japan'})
dataset = pd.get_dummies(dataset, columns=['Origin'], prefix='', prefix_sep='')

# Split the dataset
train_dataset = dataset.sample(frac=0.8, random_state=0)
test_dataset = dataset.drop(train_dataset.index)

train_features = train_dataset.copy()
test_features = test_dataset.copy()

train_labels = train_features.pop('MPG')
test_labels = test_features.pop('MPG')

After preparing the dataset, we define functions to extract them.

OSQL arrays have the same memory structure as numpy arrays, which makes importing this format efficient. Write the extraction functions in python to return a numpy array:

def get_horsepower_features():
    horsepower = np.array(train_features['Horsepower'])
    return horsepower

In dataset.osql, the foreign function is defined like this:

create function dataset:horsepower_features() -> array
as foreign 'py:fuel_efficiency.dataset.get_horsepower_features';

Note how the python function is identified: py:<modelname>.<pythonfilename>.<pythonfunctionname>. Since the memory structure is the same, the OSQL function can return an OSQL array while the python function returns a numpy array.

To run the python data pre-processing and get the horsepower feature in OSQL, call the OSQL function dataset:horsepower_features().

The rest of the dataset is defined in a similar way. SA Engine assumes the array to have C order of indexes, which is default for numpy. In pandas however, Fortran order may be used, so for two-dimensional arrays, ensure they are of C order.

def get_all_features():
    return np.array(train_features).copy(order='C').astype('float32')

create function dataset:all_features() -> array
as foreign 'py:fuel_efficiency.dataset.get_all_features';

3.3 Training

To do the training of the model in python, a model equal to the one we use in OSQL is written. For the linear regression, this means a normalization layer and a dense layer. Training choices such as optimizer and number of epochs are done here in tensorflow in py_regression.py:

def train_one_var_model(data, labels):
    normalizer = layers.Normalization(input_shape=[1,], axis=None)
    normalizer.adapt(data)
    
    model = tf.keras.Sequential([
         normalizer,
         layers.Dense(units=1)
     ])

    model.compile(
         optimizer=tf.keras.optimizers.Adam(learning_rate=0.1),
         loss='mean_absolute_error')

    history = model.fit(
         data,
         labels,
         epochs=100,
         verbose=0,
         validation_split = 0.2)
    
    return model

This function returns a model that is set up, but waiting for data to train on. Even though the return type is a tensorflow object, we can call this function from OSQL using the pythonproxy type. In py_regression.osql:

create function py:train_one_var_model(array data, array labels) -> pythonproxy
as foreign 'py:fuel_efficiency.py_regression.train_one_var_model';

py_regression.py also contain functions that work on the tensorflow model. These can be called from OSQL using the pythonproxy object.

def infer(model, data):
   return model.predict(data)

def get_model_summary(model, include_weights):
    stream = io.StringIO()
    model.summary(print_fn=lambda x: stream.write(x + '\n'))
    summary_string = stream.getvalue()
    stream.close()
    if include_weights == True:
         for layer in model.layers:
               summary_string += "Layer: " + layer.name + "\n"
               summary_string += "   weights: " + str(layer.get_weights()[0]) + "\n"
               summary_string += "   biases: " + str(layer.get_weights()[1]) + "\n"
    return summary_string

py_regression.osql:

create function py:infer(pythonproxy tf_model, array data) -> array
as foreign 'py:fuel_efficiency.py_regression.infer';

create function py:describe(pythonproxy tf_model, boolean weights) -> charstring
as foreign 'py:fuel_efficiency.py_regression.get_model_summary';

In usage.osql it is shown how these functions are combined with the dataset funtions:

// create and train one variable linear regression model in tensorflow
set :horsepower_lin_reg = py:train_one_var_model(
    dataset:horsepower_features(),
    dataset:mpg_labels()
);

// look at the weights in the tensorflow model
py:describe(:horsepower_lin_reg, true);

// linear_regression in inference.osql is an equal implementation in OSQL
// for this small example, we can manually transfer the weights
linear_regression(
    array("f32", [104.869446]), array("f32", [1446.6993]), 
    array("f32", [[-6.360543]]), array("f32", [22.58268]),  
    array("f32", [dataset:horsepower_test_features()[1]])
);

// compare with inference in python 
py:infer(
    :horsepower_lin_reg, 
    array("f32", [dataset:horsepower_test_features()[1]])
);

The same principle can be applied to larger models, for example my_dnn, the model with three dense layers and relu activation.

def train_dnn_model(data, labels):
    normalizer = layers.Normalization(axis=-1)
    normalizer.adapt(data)
    
    model = tf.keras.Sequential([
         normalizer,
         layers.Dense(64, activation='relu'),
         layers.Dense(64, activation='relu'),
         layers.Dense(1)
     ])

    model.compile(
         optimizer=tf.keras.optimizers.Adam(0.001),
         loss='mean_absolute_error')

    history = model.fit(
         data,
         labels,
         epochs=100,
         verbose=0,
         validation_split = 0.2)
    
    return model

3.4 Transfer weights

To transfer the weights from the tensorflow model to our SA Engine memory, the functions get_weights and get_bias are implemented in py_regression.py and py_regression.osql:

def get_weights(model, layer):
   return np.array(model.get_layer(layer).get_weights()[0])

def get_bias(model, layer):
   return np.array(model.get_layer(layer).get_weights()[1])

create function py:get_weights(pythonproxy tf_model, charstring layer_name) -> array of f32
as foreign 'py:fuel_efficiency.py_regression.get_weights';

create function py:get_bias(pythonproxy tf_model, charstring layer_name) -> array of f32
as foreign 'py:fuel_efficiency.py_regression.get_bias';

In usage.osql, an example shows how to use these functions to transfer the weights of my_dnn. Note that while the get_weight and get_bias functions return the weights in the tensorflow format [channels, units], the OSQL dense layer function expects the format [units, channels]. The function dense_w_trans is used to transform them.

set :dnn = py:train_dnn_model(
    dataset:all_features(),
    dataset:mpg_labels()
);

w("DNN", "normalization", py:get_weights(:dnn, 'normalization'));
w("DNN", "normalization_bias",py:get_bias(:dnn, 'normalization'));

w("DNN", "dense", dense_w_trans(py:get_weights(:dnn, 'dense')));
w("DNN", "dense_bias", py:get_bias(:dnn, 'dense'));

w("DNN", "dense_1", dense_w_trans(py:get_weights(:dnn, 'dense_1')));
w("DNN", "dense_1_bias", py:get_bias(:dnn, 'dense_1'));

w("DNN", "dense_2", dense_w_trans(py:get_weights(:dnn, 'dense_2')));
w("DNN", "dense_2_bias", py:get_bias(:dnn, 'dense_2'));

If we expected to retrain the model regularly, the weight tranfer could be added to a function:

create function get_dnn_weights() -> Boolean
as {
    declare pythonproxy dnn;
    set dnn = py:train_dnn_model(dataset:all_features(), dataset:mpg_labels());
    w("DNN", "normalization", py:get_weights(dnn, 'normalization'));
    w("DNN", "normalization_bias",py:get_bias(dnn, 'normalization'));
    w("DNN", "dense", dense_w_trans(py:get_weights(dnn, 'dense')));
    w("DNN", "dense_bias", py:get_bias(dnn, 'dense'));
    w("DNN", "dense_1", dense_w_trans(py:get_weights(dnn, 'dense_1')));
    w("DNN", "dense_1_bias", py:get_bias(dnn, 'dense_1'));
    w("DNN", "dense_2", dense_w_trans(py:get_weights(dnn, 'dense_2')));
    w("DNN", "dense_2_bias", py:get_bias(dnn, 'dense_2'));
};

Inference with the transferred weights is demonstrated in 2.2 Exporting weights to file.

my_dnn(
    w("DNN", "normalization"), w("DNN", "normalization_bias"), 
    w("DNN", "dense"), w("DNN", "dense_bias"), 
    w("DNN", "dense_1"), w("DNN", "dense_1_bias"), 
    w("DNN", "dense_2"), w("DNN", "dense_2_bias"),  
    ARRAY('f32', [8, 390, 190, 3850, 8.5, 70, 0, 0, 1])
);