# Arrays and tensors

An *multi-dimensional array* is a
collection
that represents a *tensor* whose elements are numeric values arranged
as a cube in a number of *dimensions*. For example, a numerical
vector is a 1-dimensional (1D) array, while a matrix is a 2D array of
number.

The *rank* of an array is its number of dimensions (e.g. numerical
vectors have rank 1 while matrices have rank 2), while the *shape* is
a vector containing the number of elements in each dimension. For
example, the shape if the shape of an array is `[10]`

it represents a
vector with ten elements, while the shape `[5,10]`

represents a matrix
having five rows and 10 columns.

Multi-dimensional arrays are represented as objects of type
`Array`

. The difference between objects of type `Array`

and type
`Vector`

is that type `Vector`

represent 1D sequences of objects of
any kind, while 1D arrays very efficiently represent numerical
vectors.

All elements in an array have the same data representation or
*format*. The format specifies the internal binary representation of
the elements. There is a built-in type for each subtype of `Array`

and
format. The element format is specified by strings:

Format | Internal representation | Type signature |
---|---|---|

f64/F64 | 64-bit float (Real) | Array of F64 |

f32/F32 | 32-bit float | Array of F32 |

i64/I64 | 64-bit integer (Integer) | Array of I64 |

i32/I32 | 32-bit integer | Array of I32 |

u32/U32 | 32-bit unsigned integer | Array of U32 |

i16/I16 | 16-bit integer | Array of I16 |

u16/U16 | 16-bit unsigned integer | Array of U16 |

i8/I8 | 8-bit integer | Array of I8 |

u8/U8 | 8-bit unsigned integer | Array of U8 |

## Creating arraysâ€‹

An array can be created with the `array(Vector elements)->Array`

constructor function, where `elements`

is a vector holding the
elements of the new array. The default type for arrays is 64-bit
integers (format "I64") for integer values, and 64-bit floats (format
"F64") for floating point values.

For example, the following statement constructs a 1D array of integers:

`set :a1 = array([10,20,30]);`

set :a2 = array([10.0, 20, 30, 40]);

The elements of array `:a2`

are all real numbers since
one of the array element in `array()`

call is a real number.

The function `array(Charstring format, Vector elements)->Array`

takes
an extra first argument being the format of the elements.

For example:

`set :a2 = array('I8',[1,2,3]); // a vector of 8-bit integers`

Arrays of different formats are represented as subtypes of type `Array`

,
for example:

`typesig(:a1);`

typesig(:a2)

You can create arrays of higher rank that one by nesting the
`elements`

vector.

`array([[1,2,3],`

[4,5,6]]);

To copy an array you pass the array you want to copy into the
`array(Array a)->Array`

or `array(Charstring fmt, Array a)->Array`

function.

`set :a3 = array([1,2,3]);`

set :a4 = array("U8", :a3);

set :a5 = array(format(:a4), :a4);

## Array metadataâ€‹

Arrays contain metadata describing its shape, number of elements, number of dimensions, and datatype.

The array metadata functions:

`shape(Array a)->Vector of Integer - Shape of array (e.g., [3,3] for a 3x3 array)`

dim(Array a)->Integer - Number of elements in array

rank(Array a)->Integer - Number of dimensions

format(Array a)->Charstring - Format of array elements (e.g., "F64")

length(Array a,Integer ax)->Integer - The length of dimension ax in array a

Example:

`set :d = array("I16", [[1,2,3],`

[4,5,6]]);

shape(:d);

dim(:d);

rank(:d);

format(:d);

length(:d, 2)

## Indexing arraysâ€‹

Arrays are indexed with square brackets `[]`

.

`set :arr = array([1,2,3,4,5,6,7,8,9,10]);`

:arr[3];

set :arr = array([[1,2,3],

[4,5,6]]);

:arr[1,2];

:arr[2,3];

OSQL also supports linear indexing of multidimensional arrays.

`set :arr = array([[1,2,3],`

[4,5,6]]);

:arr[5];

The character `*`

can be used to specify all values of a single dimension.

`set :arr = array([[ 1, 2, 3, 4, 5],`

[ 6, 7, 8, 9, 10],

[11, 12, 13, 14, 15],

[16, 17, 18, 19, 20]]);

:arr[1,*];

:arr[*,3];

## Arithmeticsâ€‹

Arrays support the standard mathematical operators `+`

, `-`

, `.*`

(element-wise multiplication), and `./`

(element-wise division).

`set :a = array("f32", [1,2,3,4]);`

set :b = array("f32", [-2,0.25,3,-0.1]);

:a + :b;

:a - :b;

:a .* :b;

:a ./ :b;

You can also combine scalar values with arrays:

`:a + 1;`

2 * :b;

:b / 2

## Aggregationâ€‹

Arrays can be used in aggregate functions.

Example:

`set :a = array([[1,2,3],`

[4,5,6]]);

sum(:a);

sum(:a[*,1]);

mean(:a);

## Arrays queriesâ€‹

You can specify a query that constructs a new array by an *array
comprehension* query where you specify both shape and format of the
result along with *index variables* for the elements of the new array.

For example, the following query constructs a 1D array named `out`

with shape `[5]`

indexed by the variable `i`

:

`select Array out[i..5] of U8 e`

where e = i

The array name is optional and can be omitted:

`select Array[i..5] of U8 e`

where e = i + 3;

This query constructs a 5x5 unity matrix of integers with format I8:

`select Array[i..5, j..5] of U8 e`

where e = case when i = j then 1

else 0 end;

You can use arrays in regular queries just like with all other data types.

`select array([1,2,3]);`

select out

from Array out

where out = array('U8',[1,2,3]);

## Arrays and functionsâ€‹

Derived functions can be used to create and transform arrays.

`create function my_arr() -> Array`

as array("U8", [1,2,3]);

my_arr();

You can also declare the array variable directly in the function signature.

`create function my_arr() -> Array out`

as select out

where out = array("I8", [1,2,3]) + 2;

my_arr();

The indexing variable syntax can of course be used in functions as well.

`create function inc_2d(Integer sz) -> Array`

as select Array out[y..sz, x..sz] of U8 val

where val = (y-1) + (x-1);

inc_2d(5);

Note that the array size does not have to be a scalar. It can be an expression.

`create function inc(Integer sz) -> Array`

as select Array out[i..(sz + 3)] of U8 val

where val = i;

inc(5);

### Functions with array parametersâ€‹

Derived functions can take generic arrays as input.

`create function my_fun(Array a) -> Integer`

as a[2]*10;

my_fun(array([1,2,3]));

create function my_fun2(Array a) -> Integer

as select a[i]*10

from Integer i

where i = shape(a)[1]/2;

my_fun2(array([1,2,3,4,5,6]));

Functions are type-aware, so if you use the general `Array`

output
(without the data type format) will deduce the output type from the
input.

`create function my_fun(Array a) -> Array`

as a + 2;

my_fun(array("U8",[1,2,3]));

You can also specify the format and declare array shape variables directly in the function signature.

`create function transpose(Array arr[ys,xs] of U8) -> Array`

as select Array[x..xs, y..ys] of U8 val

where val = arr[y,x];

set :in = array("U8", [[1,2,3,4],

[5,6,7,8],

[9,10,11,12]]);

transpose(:in);

OSQL also supports overloading of functions based on array format (but not on shape).

`create function overload(Array of F64 a, Array of F64 b) -> Real`

as sum(a .* b);

create function overload(Array of F32 a, Array of F32 b) -> Real

as sum(a ./ b);

overload(array("F64", [1.0,2,3]), array("F64", [4.0,5,6]));

roundto(overload(array("F32", [1.0,2,3]), array("F32", [2.0,4.0,5])), 2);

It is possible to use the same shape variable for multiple inputs. This puts constraints on the inputs requiring all shape values with the same variable to have the same value.

`create function sameshape(Array u[dim], Array v[dim])`

-> Boolean

as true;

create function is_square(Array a[dim,dim]) -> Boolean

as true;

sameshape(array([1,2,3]), array([4,5,6]));

sameshape(array([1,2,3]), array([4,5,6,7]));

is_square(array([[1,2],

[3,4]]));

is_square(array([[1,2],

[3,4],

[5,6]]));

## Reshaping arraysâ€‹

Arrays can be reshaped with the function ```
reshape(Array a,Vector
shape)->Array
```

. You can pass a desired shape, but the number of
elements have to be the same as the original.

`set :arr = array("u8", [[1, 2, 3, 4],`

[5, 6, 7, 8],

[9, 10, 11, 12],

[13, 14, 15, 16]]);

reshape(:arr, [8,2]);

You can reshape a multi-dimenasional array to a 1D numerical vector by
the function `flatten(Array)->Array`

:

`set :f = flatten(:arr);`

:f[10];

## Plotting arraysâ€‹

The default plot for arrays is `Text`

. But when the arrays represent
images you can plot the images by using `Bitmap`

plots. `Bitmap`

works
for both graylevel images (arrays with shape `[Y,X]`

) and color images
(arrays with shape `[Y,X,3]`

). Almost all array data types are
supported but all negative values will be displayed as black.

First define two functions that generate images, one graylevel and one color image:

`create function inc_2d(Integer sz) -> Array`

as select Array[y..sz, x..sz] of u8 val

where val = (y-1) + (x-1);

create function rgb_arr(Integer sz) -> Array

as select Array[y..sz,x..sz,c..3] of U8 val

where val = case when c = 1 then mod(x+y,256)

when c = 2 then mod(abs(x-y),256)

else mod((x + y)/2,256) end;

Now we can plot the generated arrays with the `//plot: Bitmap`

directive.

`//plot: Bitmap`

inc_2d(200);

`//plot: Bitmap`

rgb_arr(500);

Note that the channel order is `RGB`

(unlike, e.g., OpenCV which uses `BGR`

).

To illustrate this we can create a function that returns a "red" image:

`create function red(Integer sz) -> Array`

as select Array out[y..sz,x..sz,c..3] of U8 val

where val = case when c = 1 then mod(x + y,256)

else 0 end;

Now we can plot the red image with the `//plot: Bitmap`

directive.

`//plot: Bitmap`

red(200);