Inter-operability

Numpy Arrays

IOLink Python has capacity to inter-operate with NumPy arrays.

Python user can import its data from a NumPy Array into IOLink, and can interpret an ImageView that support the MEMORY_ACCESS capability as a NumPy array, without memory duplication.

Import a Numpy array into IOLink

IOLink python contains a class NumpyInterop which provides services for interoperability with NumPy.

import numpy
from array import array
 
# a numpy array is created
na = numpy.array([[11, 12, 13], [21, 22, 23]], numpy.uint16)
 
# a view is created on numpy array data
view = NumpyInterop.from_numpy_array(na)

Good to know, when the numpy array content is modified, the view content is impacted, and vice-versa.

Numpy allows to create arrays with different memory layouts:

• C-Layout
• Fortran-Layout

# a numpy array is created with Fortran layout
na_layout_fortran = numpy.array([[11, 12, 13], [21, 22, 23]], numpy.uint16, order ="F")
# another numpy array is created with C layout
na_layout_c = numpy.array([[11, 12, 13], [21, 22, 23]], numpy.uint16, order ="C")
 
# IOLink handles both layouts
view1 = NumpyInterop.from_numpy_array(na_layout_fortran)
view2 = NumpyInterop.from_numpy_array(na_layout_c)
 
print(view1.shape())
print(view2.shape())

Console outputs:

VectorXu64(2, 3)
VectorXu64(2, 3)

Dimension order issues

IOLink uses a canonical order of dimensions for reading and writing.

Dimensions are always ordered as following:

• COLUMN
• ROW
• SLICE
• CHANNEL
• SEQUENCE

If the Numpy array to import into IOLink does not follow the same logic, IOLink allows to configure the import mechanism as follow:

# let's suppose a Numpy array which contains 2 dimensions: ROW and COLUMN
a = numpy.array([[11, 12, 13], [21, 22, 23]], numpy.uint16)
 
# First option: tell the array describe an image
view = NumpyInterop.from_numpy_array(a, ImageTypeId_IMAGE)
 
# Second option: give the dimensions of the numpy array
view = NumpyInterop.order_from_numpy_array(a, [ImageDimension_ROW, ImageDimension_COLUMN])
 
# data are reorganized for IOLink, without impact on Numpy Array original content

By doing this, data are reorganized for IOLink, without impact on Numpy array original content, and without any copy.

Export an IOLink in-memory ImageView into an Numpy array

The export only works with ImageView with the MEMORY_ACCESS capability since data must be available in memory for Numpy. Numpy array and ImageView share the same memory. Consequently, a modification in one will impact the other.

As IOLink views and Numpy arrays use different conventions for the order of dimensions, we need to change the indexing.

In Numpy conventions, order is almost inversed (T, Z, Y, X, C) in comparison to IOLink canonical order.

In the case of a view on vectorial data, Vector3u16 for example, the Numpy array will have an additionnal dimension at the end, its size being the number of components of the vectorial data type.

Example: a VOLUME ImageView of shape (12, 4, 6) will result in a Numpy array with a shape of (6, 4, 12) if the view store a scalar data type. If the stored data type is vectorial, for example Vector3u16, the resulting Numpy array will have the following shape (6, 4, 12, 3).

# an in-memory ImageView identified as a VOLUME
type = DataType(DataTypeId.UINT16);
properties = iolink.ImageProperties.from_data_type(type);
properties.image_info.set_axes_interpretation(iolink.ImageTypeId.VOLUME)
image = ImageViewFactory.allocate((6, 4), type, properties, None)
 
# export to Numpy array
np = NumpyInterop.to_numpy_array(image)
 
print(np.shape)

Console outputs:

(4, 6)

If your ImageView has no axes interpretation, which means dimensions are not clearly identified, the reorder is not possible at conversion into Numpy. Dimensions are kept in original order. Only vectorial DataType will be converted into an additionnal dimension at the end.

# an in-memory ImageView
dtype = DataType(DataTypeId.VEC3_UINT16)
image = ImageViewFactory.allocate((6, 4), dtype)
 
# export to Numpy array
np = NumpyInterop.to_numpy_array(image)
 
print(np.shape)

Console outputs:

(6, 4, 3)

As developers can choose to not follow Numpy convention, IOLink provides another method to export an ImageView as a Numpy Array. This time, the developer can choose the order of axes during export. But it needs the ImageView has all its axes well-defined (axes interpretation should not be UNKNOWN).

# an in-memory ImageView identified as a VOLUME
dtype = DataType(DataTypeId.UINT16)
properties = iolink.ImageProperties.from_data_type(dtype)
properties.image_info.set_axes_interpretation(iolink.ImageTypeId.VOLUME)
image = ImageViewFactory.allocate((6, 5, 4), dtype, properties, None)
 
# export to Numpy array by indicating axes order
np = NumpyInterop.order_to_numpy_array(image, [ROW, SLICE, COLUMN])
 
print(np.shape)

Console outputs:

(5, 4, 6)

In the case of vectorial data, a CHANNEL dimension will be created at export and shall be also ordered:

# an in-memory ImageView identified as a VOLUME
dtype = DataType(DataTypeId.VEC3_UINT16)
properties = iolink.ImageProperties.from_data_type(dtype)
properties.image_info.set_axes_interpretation(iolink.ImageTypeId.VOLUME)
image = ImageViewFactory.allocate((6, 5, 4), dtype, properties, None)
 
# export to Numpy array by indicating axes order
np = NumpyInterop.order_to_numpy_array(image, [ROW, SLICE, CHANNEL, COLUMN])
 
print(np.shape)

Console outputs:

(5, 4, 3, 6)

RAW I/O

In io module, I/O classes provides APIs to manipulate data streams.

StreamAccess in IOLink allows to manipulate raw data and is quite similar with the abstract class io.RawIOBase (which inherits from io.IOBase.

IOLink provides an inter-operability mechanism to adapt StreamAccess as a io.RawIOBase object.

stream = iolink.StreamAccessFactory.allocate()
ioBaseStream = stream.to_raw_io()
isinstance(ioBaseStream, io.RawIOBase)

Console outputs:

True