IOLink provides a mechanism of serialization for most of its objects. It allows to encode any object into a StreamAccess, and then, decode them to re-build similar objects in memory.

For each object, an encoding format has been defined, but a header is common to all objects.

Encoding format is versioned to check compatibility at decoding. A major and a minor value are encoded in the header.

Only minor versions are compatible.

To avoid any endianness issue, all data are encoded in big-endian and decoded in native-endian.

Common Header

Starts every IOLink object serialization.

Offset	Size	Type	Information
0	4	uint32_t	Magic number "IOL " (ASCII encoding)
4	2	uint16_t	Serialization major version
6	2	uint16_t	Serialization minor version
8	4	uint32_t	Object identifier (see each object formatting)

Header is only written once for a whole object, and is not repeated for each internal object encoding.

e.g.: Specifically encoding a Vector triggers the writing of a header. But to encode an ImageView (which contains many Vectors), headers for Vector objects are skipped.

Version compatibility

In the header, a serialization version field indicates which version was used for encoding. When decoding, this version is checked to ensure compatibility:

encoding version	decoding version	compatibility
X.Y	X.Y	compatible
X.Y	X.Z	compatible if Z > Y
X.Y	W.Z	not compatible

Internally, in a newer minor version, only optional fields can be added to an existing object (already defined object identifier). A new major version is defined if new object are created or for any other modifications in existing ones.

Data section

Generic data are always encoded the same and as follows:

Offset	Size	Type	Information
0	8	uint64_t	Size of data (in bytes)
8	see previous field	uint8_t	data

String section

String are encoded as follows:

Offset	Size	Type	Information
0	8	uint64_t	Length of string in bytes - no null-character at termination
8	Data section size	Data section	string content

Vector and VectorX

Vector and VectorX, despite they are 2 different objects in IOLink, have the same encoding format so as to ease the conversion from one to the other during decoding.

Object identifier
"VECT" (ASCII encoding)

Vector and VectorX are encoded as follow:

Offset (from header)	Size	Type	Information
0	8	uint64_t	Scalar type identifier (see DataType)
8	8	uint64_t	Element count in the Vector(X) (is redundant with Type id for Vector objects)

At offset 16, vector components are encoded into a data section (See Data section).

For a Vector4u64 containing values [5, 15, 25, 35], elements are encoded in following order:

first 64-bit-word: 5
second 64-bit-word : 15
third 64-bit-word: 25
fourth 64-bit-word: 35

Here is an example of serialization of a Vector:

// we use an in-memory stream, but it can be any StreamAccess
auto stream = StreamAccessFactory::allocate();
 
Vector2u64 vect{50, 125};
 
// encode input Vector as Vector
Serialization::encodeVector2u64(vectIn, stream);

And then the decoding (in the meanwhile, stream object was reset to its initial position):

// decode the expected Vector2u64 from the stream
auto vect = Serialization::decodeVector2u64(stream);
 
// or decode the Vector2u64 as a VectorXu64
// auto vect = Serialization::decodeVectorXu64(stream);

ImageView

ImageViews can be fully serialized. At decoding, an in-memory copy of original ImageView will be created.

A header with following OID precedes the encoding:

Object IDentifier
"IMAG" (ASCII encoding)

ImageView is encoded as follow:

Offset (from header)	Size	Type	Information
0	depends on dimension count	VectorXu64	shape of the ImageView (see VectorX)
undefined	8	uint64_t	datatype identifier (see DataType)
undefined	size of ImageProperties section	ImageProperties	ImageProperties of ImageView (see ImageProperties)
undefined	size of Metadata section	Metadata	Metadata tree of ImageView (see Metadata)
undefined	image Size x DataType size	uint8_t	image data (See Data)

ImageProperties

Offset	Size	Type	Information
0	size of SpatialCalibration section	SpatialCalibration	SpatialCalibration of image (see SpatialCalibration)
undefined	size of ImageInfoProperties section	ImageInfoProperties	ImageInfoProperties of image (see ImageInfoProperties)

SpatialCalibration

Offset	Size	Type	Information
0	24	Vector3d	origin of image (see VectorX)
24	24	Vector3d	spacing of image (see VectorX)
48	24	Vector3d	direction column of image (see VectorX)
72	24	Vector3d	direction row of image (see VectorX)
96	size of string section	String	unit (see String)

ImageInfoProperties

Offset	Size	Type	Information
0	8	uint64_t	ImageType
8	8	uint64_t	ImageInterpretation
16	8	uint64_t	hasAlpha
24	8	uint64_t	bitDepth
32	16	Vector2d	Value Range (see VectorX)

Metadata

Metadata is an optional field of ImageView (can be null). Thus, each Metadata node section starts with a 64-bit-word indicating the presence of a node or not.

Tag value	Description
"NMET" (ASCII encoding)	No node
"META" (ASCII encoding)	a node is encoded

And then, node is encoded as follows:

Offset after Tag	Size	Type	Information
0	Size of string section	String	Key of node
undefined	Size of VariantDataValue section	VariantDataValue	Value of node (see VariantDataValue)
undefined	8	uint64_t	Child count
undefined	Size of Metadata section	Metadata	Child 1 (if exists)
undefined	Size of Metadata section	Metadata	Child 2 (if exists)
undefined	Size of Metadata section	Metadata	Child ...(if exists)

For example, if we have the following Metadata tree:

Example of Metadata tree

it will be stored in this following order:

Metadata
root
MetadataNode1
MetadataNode4
MetadataNode2
MetadataNode5
MetadataNode8
MetadataNode9
MetadataNode6
MetadataNode7
MetadataNode3

VariantDataValue

Value of a node is an optional field (can be null). Thus, each VariantDataValue section starts with 64-bit-word indicating the presence of a value or not.

Tag value	Description
"NVAR" (ASCII encoding)	No value
"VARI" (ASCII encoding)	a value is encoded

And then, VariantDataValue is encoded as follows:

Offset after Tag	Size	Type	Information
0	8	uint64_t	DataType ident of value (see DataType)
8	8	uint64_t	value count
16	Data section size	DataSection	list of values (See Data section)

Here is an example of encoding and decoding of an ImageView:

// we use an in-memory stream, but it can be any StreamAccess
auto stream = StreamAccessFactory::allocate();
 
auto image = ImageViewFactory::allocate(VectorXu64{10, 11, 12}, DataTypeId::INT8);
 
// encode input ImageView
Serialization::encodeImageView(image, stream);
 
// reset stream
stream->seek(0, SeekOrigin::BEGIN);
 
// decode input ImageView
auto imageDecoded = Serialization::decodeImageView(stream);

ArrayX

A header with the following OID precedes the encoding:

Object IDentifier
"ARRY" (ASCII encoding)

ArrayX is encoded as follows:

Offset (from header)	Size	Type	Information
0	variable	VectorXu64	shape of the ArrayX (see VectorX)
undefined	8	uint64_t	datatype identifier (see DataType)
undefined	array size x data type size	uint8_t	image data (See Data)

DataFrameView

A header with the following OID precedes the encoding:

Object IDentifier
"DATF" (ASCII encoding)

DataFrameViews are encoded as follows:

Offset (from header)	Size	Type	Information
0	16	Vector2u64	shape of the DataFrameView (see Vector)
16	size of string section	String	name of column #0 (see String)
undefined	8	uint64_t	datatype identifier for column #0 (see DataType)
undefined	size of string section	String	name of column #1 (see String)
undefined	8	uint64_t	datatype identifier for column #1 (see DataType)
undefined	size of string section	String	name of column #... (see String)
undefined	8	uint64_t	datatype identifier for column #... (see DataType)
undefined	size of string section	String	name of column #N (see String)
undefined	8	uint64_t	datatype identifier for column #N (see DataType)
undefined	DataFrame data section size	uint8_t	column #0 data (See Data)
undefined	DataFrame data section size	uint8_t	column #1 data (See Data)
undefined	DataFrame data section size	uint8_t	column #... data (See Data)
undefined	DataFrame data section size	uint8_t	column #N data (See Data)

DataFrame data

DataFrameView column data can be either a list of strings or numerical values. Consequently column data can be encoded in a classical data section or in a String section, depending on DataType of each column.

Matrix

A header with the following OID precedes the encoding:

Object IDentifier
"MATR" (ASCII encoding)

Matrix is encoded as follows:

Offset (from header)	Size	Type	Information
0	8	uint64_t	dimension N of Matrix (3 or 4)
8	8	uint64_t	datatype identifier (see DataType)
16	N x N x data type size	uint8_t	matrix data (See Data)