Arrays and tensors

This page uses Wasm code blocks so you can run the examples directly in the browser.

An multi-dimensional array is a collection that represents a tensor whose elements are numeric values arranged as a cube in a number of dimensions. For example, a numerical vector is a 1-dimensional (1D) array, while a matrix is a 2D array of number.

The rank of an array is its number of dimensions (e.g. numerical vectors have rank 1 while matrices have rank 2), while the shape is a vector containing the number of elements in each dimension. For example, the shape if the shape of an array is [10] it represents a vector with ten elements, while the shape [5,10] represents a matrix having five rows and 10 columns.

Multi-dimensional arrays are represented as objects of type Array. The difference between objects of type Array and type Vector is that type Vector represent 1D sequences of objects of any kind, while 1D arrays very efficiently represent numerical vectors.

All elements in an array have the same data representation or format. The format specifies the internal binary representation of the elements. There is a built-in type for each subtype of Array and format. The element format is specified by strings:

Format	Internal representation	Type signature
f64/F64	64-bit float (Real)	Array of F64
f32/F32	32-bit float	Array of F32
i64/I64	64-bit integer (Integer)	Array of I64
i32/I32	32-bit integer	Array of I32
u32/U32	32-bit unsigned integer	Array of U32
i16/I16	16-bit integer	Array of I16
u16/U16	16-bit unsigned integer	Array of U16
i8/I8	8-bit integer	Array of I8
u8/U8	8-bit unsigned integer	Array of U8

Creating arrays

An array can be created with the array(Vector elements)->Array constructor function, where elements is a vector holding the elements of the new array. The default type for arrays is 64-bit integers (format "I64") for integer values, and 64-bit floats (format "F64") for floating point values.

For example, the following statement constructs a 1D array of integers:

set :a1 = array([10,20,30]);

set :a2 = array([10.0, 20, 30, 40]);

The elements of array :a2 are all real numbers since one of the array element in array() call is a real number.

The function array(Charstring format, Vector elements)->Array takes an extra first argument being the format of the elements.

For example:

set :a2 = array('I8',[1,2,3]); // a vector of 8-bit integers

Arrays of different formats are represented as subtypes of type Array, for example:

typesig(:a1);

typesig(:a2)

You can create arrays of higher rank that one by nesting the elements vector.

array([[1,2,3],
       [4,5,6]]);

To copy an array you pass the array you want to copy into the array(Array a)->Array or array(Charstring fmt, Array a)->Arrayfunction.

set :a3 = array([1,2,3]);

set :a4 = array("U8", :a3);

set :a5 = array(format(:a4), :a4);

Array metadata

Arrays contain metadata describing its shape, number of elements, number of dimensions, and datatype.

The array metadata functions:

shape(Array a)->Vector of Integer        - Shape of array (e.g., [3,3] for a 3x3 array)
dim(Array a)->Integer                    - Number of elements in array 
rank(Array a)->Integer                   - Number of dimensions
format(Array a)->Charstring              - Format of array elements (e.g., "F64")
length(Array a,Integer ax)->Integer      - The length of dimension ax in array a

Example:

set :d = array("I16", [[1,2,3],
                       [4,5,6]]);

shape(:d);

dim(:d);

rank(:d);

format(:d);

length(:d, 2)

Indexing arrays

Arrays are indexed with square brackets [].

set :arr = array([1,2,3,4,5,6,7,8,9,10]);

:arr[3];

set :arr = array([[1,2,3],
                  [4,5,6]]);

:arr[1,2];

:arr[2,3];

OSQL also supports linear indexing of multidimensional arrays.

set :arr = array([[1,2,3],
                  [4,5,6]]);

:arr[5];

The character * can be used to specify all values of a single dimension.

set :arr = array([[ 1,  2,  3,  4,  5],
                  [ 6,  7,  8,  9, 10],
                  [11, 12, 13, 14, 15],
                  [16, 17, 18, 19, 20]]);

:arr[1,*];


:arr[*,3];

Arithmetics

Arrays support the standard mathematical operators +, -, .* (element-wise multiplication), and ./ (element-wise division).

set :a = array("f32", [1,2,3,4]);
set :b = array("f32", [-2,0.25,3,-0.1]);

:a + :b;

:a - :b;

:a .* :b;

:a ./ :b;

You can also combine scalar values with arrays:

:a + 1;

2 * :b;

:b / 2

Aggregation

Arrays can be used in aggregate functions.

Example:

set :a = array([[1,2,3],
                [4,5,6]]);

sum(:a);

sum(:a[*,1]);

mean(:a);

Arrays queries

You can specify a query that constructs a new array by an array comprehension query where you specify both shape and format of the result along with index variables for the elements of the new array.

For example, the following query constructs a 1D array named out with shape [5] indexed by the variable i:

select Array out[i..5] of U8 e
 where e = i

The array name is optional and can be omitted:

select Array[i..5] of U8 e
 where e = i + 3;

This query constructs a 5x5 unity matrix of integers with format I8:

select Array[i..5, j..5] of U8 e
 where e = case when i = j then 1
                else 0 end;

You can use arrays in regular queries just like with all other data types.

select array([1,2,3]);

select out
  from Array out
 where out = array('U8',[1,2,3]);

Arrays and functions

Derived functions can be used to create and transform arrays.

create function my_arr() -> Array
  as array("U8", [1,2,3]);

my_arr();

You can also declare the array variable directly in the function signature.

create function my_arr() -> Array out
  as select out
      where out = array("I8", [1,2,3]) + 2;

my_arr();

The indexing variable syntax can of course be used in functions as well.

create function inc_2d(Integer sz) -> Array
  as select Array out[y..sz, x..sz] of U8 val
      where val = (y-1) + (x-1);

inc_2d(5);

Note that the array size does not have to be a scalar. It can be an expression.

create function inc(Integer sz) -> Array
  as select Array out[i..(sz + 3)] of U8 val
      where val = i;

inc(5);

Functions with array parameters

Derived functions can take generic arrays as input.

create function my_fun(Array a) -> Integer
  as a[2]*10;

my_fun(array([1,2,3]));

create function my_fun2(Array a) -> Integer
  as select a[i]*10
      from Integer i
     where i = shape(a)[1]/2;

my_fun2(array([1,2,3,4,5,6]));

Functions are type-aware, so if you use the general Array output (without the data type format) will deduce the output type from the input.

create function my_fun(Array a) -> Array
  as a + 2;

my_fun(array("U8",[1,2,3]));

You can also specify the format and declare array shape variables directly in the function signature.

create function transpose(Array arr[ys,xs] of U8) -> Array
  as select Array[x..xs, y..ys] of U8 val
      where val = arr[y,x];

set :in = array("U8", [[1,2,3,4],
                       [5,6,7,8],
                       [9,10,11,12]]);

transpose(:in);

OSQL also supports overloading of functions based on array format (but not on shape).

create function overload(Array of F64 a, Array of F64 b) -> Real
  as sum(a .* b);

create function overload(Array of F32 a, Array of F32 b) -> Real
  as sum(a ./ b);

overload(array("F64", [1.0,2,3]), array("F64", [4.0,5,6]));

roundto(overload(array("F32", [1.0,2,3]), array("F32", [2.0,4.0,5])), 2);

It is possible to use the same shape variable for multiple inputs. This puts constraints on the inputs requiring all shape values with the same variable to have the same value.

create function sameshape(Array u[dim], Array v[dim])
                        -> Boolean
   as true;

create function is_square(Array a[dim,dim]) -> Boolean
   as true;

sameshape(array([1,2,3]), array([4,5,6]));

sameshape(array([1,2,3]), array([4,5,6,7]));

is_square(array([[1,2],
                 [3,4]]));

is_square(array([[1,2],
                 [3,4],
                 [5,6]]));

Reshaping arrays

Arrays can be reshaped with the function reshape(Array a,Vector shape)->Array. You can pass a desired shape, but the number of elements have to be the same as the original.

set :arr = array("u8", [[1,  2,  3,  4],
                        [5,  6,  7,  8],
                        [9, 10, 11, 12],
                        [13, 14, 15, 16]]);

reshape(:arr, [8,2]);

You can reshape a multi-dimenasional array to a 1D numerical vector by the function flatten(Array)->Array:

set :f = flatten(:arr);

:f[10];

Plotting arrays

The default plot for arrays is Text. But when the arrays represent images you can plot the images by using Bitmap plots. Bitmap works for both graylevel images (arrays with shape [Y,X]) and color images (arrays with shape [Y,X,3]). Almost all array data types are supported but all negative values will be displayed as black.

First define two functions that generate images, one graylevel and one color image:

create function inc_2d(Integer sz) -> Array
  as select Array[y..sz, x..sz] of u8 val
      where val = (y-1) + (x-1);

create function rgb_arr(Integer sz) -> Array
  as select Array[y..sz,x..sz,c..3] of U8 val
      where val = case when c = 1 then mod(x+y,256)
                       when c = 2 then mod(abs(x-y),256)
                       else mod((x + y)/2,256) end;

Now we can plot the generated arrays with the //plot: Bitmap directive.

//plot: Bitmap
inc_2d(200);

//plot: Bitmap
rgb_arr(500);

Note that the channel order is RGB (unlike, e.g., OpenCV which uses BGR).

To illustrate this we can create a function that returns a "red" image:

create function red(Integer sz) -> Array
  as select Array out[y..sz,x..sz,c..3] of U8 val
      where val = case when c = 1 then mod(x + y,256)
                       else 0 end;

Now we can plot the red image with the //plot: Bitmap directive.

//plot: Bitmap
red(200);

Functions

Array functions

Arrays and tensors

Creating arrays​

Array metadata​

Indexing arrays​

Arithmetics​

Aggregation​

Arrays queries​

Arrays and functions​

Functions with array parameters​

Reshaping arrays​

Plotting arrays​

Functions​