ImageDev

OnnxPredictionFiltering3d

Computes a prediction on a three-dimensional image from an ONNX model and generates an image representing the prediction scores.

Access to parameter description

For an overview, please refer to the Deep Learning section.
This algorithm produces an image containing the raw prediction scores given by the model. Depending on how the model has been trained, it can be used either to perform image filtering, or segmentation by applying an appropriate post-processing step afterwards.

The following steps are applied:
See also

Function Syntax

This function returns outputImage. 
// Function prototype
std::shared_ptr< iolink::ImageView >
onnxPredictionFiltering3d( std::shared_ptr< iolink::ImageView > inputImage,
                           std::string modelPath,
                           OnnxPredictionFiltering3d::DataFormat dataFormat,
                           OnnxPredictionFiltering3d::InputNormalizationType inputNormalizationType,
                           iolink::Vector2d normalizationRange,
                           OnnxPredictionFiltering3d::NormalizationScope normalizationScope,
                           iolink::Vector3u32 tileSize,
                           uint32_t tileOverlap,
                           OnnxPredictionFiltering3d::OutputNormalizationType outputNormalizationType,
                           OnnxPredictionFiltering3d::OutputType outputType,
                           std::shared_ptr< iolink::ImageView > outputImage = nullptr );
This function returns outputImage. 
// Function prototype.
onnx_prediction_filtering_3d(input_image: idt.ImageType,
                             model_path: str = "",
                             data_format: OnnxPredictionFiltering3d.DataFormat = OnnxPredictionFiltering3d.DataFormat.NDHWC,
                             input_normalization_type: OnnxPredictionFiltering3d.InputNormalizationType = OnnxPredictionFiltering3d.InputNormalizationType.STANDARDIZATION,
                             normalization_range: Union[Iterable[int], Iterable[float]] = [0, 1],
                             normalization_scope: OnnxPredictionFiltering3d.NormalizationScope = OnnxPredictionFiltering3d.NormalizationScope.GLOBAL,
                             tile_size: Iterable[int] = [128, 128, 128],
                             tile_overlap: int = 16,
                             output_normalization_type: OnnxPredictionFiltering3d.OutputNormalizationType = OnnxPredictionFiltering3d.OutputNormalizationType.NONE,
                             output_type: OnnxPredictionFiltering3d.OutputType = OnnxPredictionFiltering3d.OutputType.SAME_AS_INPUT,
                             output_image: idt.ImageType = None) -> idt.ImageType
This function returns outputImage. 
// Function prototype.
public static IOLink.ImageView
OnnxPredictionFiltering3d( IOLink.ImageView inputImage,
                           String modelPath = "",
                           OnnxPredictionFiltering3d.DataFormat dataFormat = ImageDev.OnnxPredictionFiltering3d.DataFormat.NDHWC,
                           OnnxPredictionFiltering3d.InputNormalizationType inputNormalizationType = ImageDev.OnnxPredictionFiltering3d.InputNormalizationType.STANDARDIZATION,
                           double[] normalizationRange = null,
                           OnnxPredictionFiltering3d.NormalizationScope normalizationScope = ImageDev.OnnxPredictionFiltering3d.NormalizationScope.GLOBAL,
                           uint[] tileSize = null,
                           UInt32 tileOverlap = 16,
                           OnnxPredictionFiltering3d.OutputNormalizationType outputNormalizationType = ImageDev.OnnxPredictionFiltering3d.OutputNormalizationType.NONE,
                           OnnxPredictionFiltering3d.OutputType outputType = ImageDev.OnnxPredictionFiltering3d.OutputType.SAME_AS_INPUT,
                           IOLink.ImageView outputImage = null );

Class Syntax

Parameters

Parameter Name Description Type Supported Values Default Value
input
inputImage
The input image. It can be a grayscale or color image, depending on the selected model. Image Binary, Label, Grayscale or Multispectral nullptr
input
modelPath
The path to the ONNX model file. String ""
input
dataFormat
The tensor layout expected as input by the model. The input image is automatically converted to this layout by the algorithm.
NDHWC The layout is organized with interlaced channels. For instance, if the input is a color image, each pixel presents its RGB components successively.
NCDHW The layout is organized with separate channels. Each channel is an individual plan.
Enumeration NDHWC
input
inputNormalizationType
The type of normalization to apply before computing the prediction. It is recommended to apply the same pre-processing as during the training.
NONE No normalization is applied before executing the prediction.
STANDARDIZATION A normalization is applied by subtracting the mean and dividing by the standard deviation.
MIN_MAX A normalization is applied by subtracting the minimum and dividing by data range.
Enumeration STANDARDIZATION
input
normalizationRange
The data range in which the input image is normalized before computing the prediction. It is recommended to apply the same pre-processing as during the training. This parameter is ignored if the normalization type is set to NONE. Vector2d Any value {0.f, 1.f}
input
normalizationScope
The scope for computing normalization (mean, standard deviation, minimum or maximum). This parameter is ignored if the normalization type is set to NONE.
GLOBAL The normalization is applied globally on the input batch.
PER_VOLUME The normalization is applied individually on each image of the input batch.
Enumeration GLOBAL
input
tileSize
The width, height, and depth in pixels of the sliding window. This size includes the user defined tile overlap. It must be a multiple of 2 to the power of the number of downsampling or upsampling layers.
Guidelines to select an appropriate tile size are available in the Tiling section. 		Vector3u32 != 0 {128, 128, 128}
input
tileOverlap
The number of pixels used as overlap between the tiles. An overlap of zero may lead to artifacts in the prediction result. A non-zero overlap reduces such artifacts but increases the computation time. UInt32 Any value 16
input
outputNormalizationType
The type of normalization to apply after computing the prediction. This parameter is ignored if the input normalization type is set to NONE.
NONE No normalization is applied on the output image.
INVERSE The inverse operation is applied to set the output image in the same range as the input.
Enumeration NONE
input
outputType
The output data type. It can either be the same as the input type or forced to float. This parameter is ignored if the input normalization type is set to NONE.
SAME_AS_INPUT The output image has the same type as the input image.
FLOAT_32_BIT The output image type is forced to floating point.
Enumeration SAME_AS_INPUT
output
outputImage
The output image. Its spatial dimensions, and calibration are forced to the same values as the input. Its number of channels depends on the selected model. Its type depends on the selected output type. Image nullptr
Parameter Name Description Type Supported Values Default Value
input
input_image
The input image. It can be a grayscale or color image, depending on the selected model. image Binary, Label, Grayscale or Multispectral None
input
model_path
The path to the ONNX model file. string ""
input
data_format
The tensor layout expected as input by the model. The input image is automatically converted to this layout by the algorithm.
NDHWC The layout is organized with interlaced channels. For instance, if the input is a color image, each pixel presents its RGB components successively.
NCDHW The layout is organized with separate channels. Each channel is an individual plan.
enumeration NDHWC
input
input_normalization_type
The type of normalization to apply before computing the prediction. It is recommended to apply the same pre-processing as during the training.
NONE No normalization is applied before executing the prediction.
STANDARDIZATION A normalization is applied by subtracting the mean and dividing by the standard deviation.
MIN_MAX A normalization is applied by subtracting the minimum and dividing by data range.
enumeration STANDARDIZATION
input
normalization_range
The data range in which the input image is normalized before computing the prediction. It is recommended to apply the same pre-processing as during the training. This parameter is ignored if the normalization type is set to NONE. vector2d Any value [0, 1]
input
normalization_scope
The scope for computing normalization (mean, standard deviation, minimum or maximum). This parameter is ignored if the normalization type is set to NONE.
GLOBAL The normalization is applied globally on the input batch.
PER_VOLUME The normalization is applied individually on each image of the input batch.
enumeration GLOBAL
input
tile_size
The width, height, and depth in pixels of the sliding window. This size includes the user defined tile overlap. It must be a multiple of 2 to the power of the number of downsampling or upsampling layers.
Guidelines to select an appropriate tile size are available in the Tiling section. 		vector3u32 != 0 [128, 128, 128]
input
tile_overlap
The number of pixels used as overlap between the tiles. An overlap of zero may lead to artifacts in the prediction result. A non-zero overlap reduces such artifacts but increases the computation time. uint32 Any value 16
input
output_normalization_type
The type of normalization to apply after computing the prediction. This parameter is ignored if the input normalization type is set to NONE.
NONE No normalization is applied on the output image.
INVERSE The inverse operation is applied to set the output image in the same range as the input.
enumeration NONE
input
output_type
The output data type. It can either be the same as the input type or forced to float. This parameter is ignored if the input normalization type is set to NONE.
SAME_AS_INPUT The output image has the same type as the input image.
FLOAT_32_BIT The output image type is forced to floating point.
enumeration SAME_AS_INPUT
output
output_image
The output image. Its spatial dimensions, and calibration are forced to the same values as the input. Its number of channels depends on the selected model. Its type depends on the selected output type. image None
Parameter Name Description Type Supported Values Default Value
input
inputImage
The input image. It can be a grayscale or color image, depending on the selected model. Image Binary, Label, Grayscale or Multispectral null
input
modelPath
The path to the ONNX model file. String ""
input
dataFormat
The tensor layout expected as input by the model. The input image is automatically converted to this layout by the algorithm.
NDHWC The layout is organized with interlaced channels. For instance, if the input is a color image, each pixel presents its RGB components successively.
NCDHW The layout is organized with separate channels. Each channel is an individual plan.
Enumeration NDHWC
input
inputNormalizationType
The type of normalization to apply before computing the prediction. It is recommended to apply the same pre-processing as during the training.
NONE No normalization is applied before executing the prediction.
STANDARDIZATION A normalization is applied by subtracting the mean and dividing by the standard deviation.
MIN_MAX A normalization is applied by subtracting the minimum and dividing by data range.
Enumeration STANDARDIZATION
input
normalizationRange
The data range in which the input image is normalized before computing the prediction. It is recommended to apply the same pre-processing as during the training. This parameter is ignored if the normalization type is set to NONE. Vector2d Any value {0f, 1f}
input
normalizationScope
The scope for computing normalization (mean, standard deviation, minimum or maximum). This parameter is ignored if the normalization type is set to NONE.
GLOBAL The normalization is applied globally on the input batch.
PER_VOLUME The normalization is applied individually on each image of the input batch.
Enumeration GLOBAL
input
tileSize
The width, height, and depth in pixels of the sliding window. This size includes the user defined tile overlap. It must be a multiple of 2 to the power of the number of downsampling or upsampling layers.
Guidelines to select an appropriate tile size are available in the Tiling section. 		Vector3u32 != 0 {128, 128, 128}
input
tileOverlap
The number of pixels used as overlap between the tiles. An overlap of zero may lead to artifacts in the prediction result. A non-zero overlap reduces such artifacts but increases the computation time. UInt32 Any value 16
input
outputNormalizationType
The type of normalization to apply after computing the prediction. This parameter is ignored if the input normalization type is set to NONE.
NONE No normalization is applied on the output image.
INVERSE The inverse operation is applied to set the output image in the same range as the input.
Enumeration NONE
input
outputType
The output data type. It can either be the same as the input type or forced to float. This parameter is ignored if the input normalization type is set to NONE.
SAME_AS_INPUT The output image has the same type as the input image.
FLOAT_32_BIT The output image type is forced to floating point.
Enumeration SAME_AS_INPUT
output
outputImage
The output image. Its spatial dimensions, and calibration are forced to the same values as the input. Its number of channels depends on the selected model. Its type depends on the selected output type. Image null

Object Examples

auto chocolate_bar = readVipImage( std::string( IMAGEDEVDATA_IMAGES_FOLDER ) + "chocolate_bar.vip" );

OnnxPredictionFiltering3d onnxPredictionFiltering3dAlgo;
onnxPredictionFiltering3dAlgo.setInputImage( chocolate_bar );
onnxPredictionFiltering3dAlgo.setModelPath( std::string( IMAGEDEVDATA_OBJECTS_FOLDER ) + "segmentation3d.onnx" );
onnxPredictionFiltering3dAlgo.setDataFormat( OnnxPredictionFiltering3d::DataFormat::NDHWC );
onnxPredictionFiltering3dAlgo.setInputNormalizationType( OnnxPredictionFiltering3d::InputNormalizationType::NONE );
onnxPredictionFiltering3dAlgo.setNormalizationRange( {0, 1} );
onnxPredictionFiltering3dAlgo.setNormalizationScope( OnnxPredictionFiltering3d::NormalizationScope::GLOBAL );
onnxPredictionFiltering3dAlgo.setTileSize( {128, 128, 128} );
onnxPredictionFiltering3dAlgo.setTileOverlap( 16 );
onnxPredictionFiltering3dAlgo.setOutputNormalizationType( OnnxPredictionFiltering3d::OutputNormalizationType::NONE );
onnxPredictionFiltering3dAlgo.setOutputType( OnnxPredictionFiltering3d::OutputType::SAME_AS_INPUT );
onnxPredictionFiltering3dAlgo.execute();

std::cout << "outputImage:" << onnxPredictionFiltering3dAlgo.outputImage()->toString();

chocolate_bar = imagedev.read_vip_image(imagedev_data.get_image_path("chocolate_bar.vip"))

onnx_prediction_filtering_3d_algo = imagedev.OnnxPredictionFiltering3d()
onnx_prediction_filtering_3d_algo.input_image = chocolate_bar
onnx_prediction_filtering_3d_algo.model_path = imagedev_data.get_object_path("segmentation3d.onnx")
onnx_prediction_filtering_3d_algo.data_format = imagedev.OnnxPredictionFiltering3d.NDHWC
onnx_prediction_filtering_3d_algo.input_normalization_type = imagedev.OnnxPredictionFiltering3d.InputNormalizationType.NONE
onnx_prediction_filtering_3d_algo.normalization_range = [0, 1]
onnx_prediction_filtering_3d_algo.normalization_scope = imagedev.OnnxPredictionFiltering3d.GLOBAL
onnx_prediction_filtering_3d_algo.tile_size = [128, 128, 128]
onnx_prediction_filtering_3d_algo.tile_overlap = 16
onnx_prediction_filtering_3d_algo.output_normalization_type = imagedev.OnnxPredictionFiltering3d.OutputNormalizationType.NONE
onnx_prediction_filtering_3d_algo.output_type = imagedev.OnnxPredictionFiltering3d.SAME_AS_INPUT
onnx_prediction_filtering_3d_algo.execute()

print("output_image:", str(onnx_prediction_filtering_3d_algo.output_image))

ImageView chocolate_bar = Data.ReadVipImage( @"Data/images/chocolate_bar.vip" );

OnnxPredictionFiltering3d onnxPredictionFiltering3dAlgo = new OnnxPredictionFiltering3d
{
    inputImage = chocolate_bar,
    modelPath = @"Data/objects/segmentation3d.onnx",
    dataFormat = OnnxPredictionFiltering3d.DataFormat.NDHWC,
    inputNormalizationType = OnnxPredictionFiltering3d.InputNormalizationType.NONE,
    normalizationRange = new double[]{0, 1},
    normalizationScope = OnnxPredictionFiltering3d.NormalizationScope.GLOBAL,
    tileSize = new uint[]{128, 128, 128},
    tileOverlap = 16,
    outputNormalizationType = OnnxPredictionFiltering3d.OutputNormalizationType.NONE,
    outputType = OnnxPredictionFiltering3d.OutputType.SAME_AS_INPUT
};
onnxPredictionFiltering3dAlgo.Execute();

Console.WriteLine( "outputImage:" + onnxPredictionFiltering3dAlgo.outputImage.ToString() );

Function Examples

auto chocolate_bar = readVipImage( std::string( IMAGEDEVDATA_IMAGES_FOLDER ) + "chocolate_bar.vip" );

auto result = onnxPredictionFiltering3d( chocolate_bar, std::string( IMAGEDEVDATA_OBJECTS_FOLDER ) + "segmentation3d.onnx", OnnxPredictionFiltering3d::DataFormat::NDHWC, OnnxPredictionFiltering3d::InputNormalizationType::NONE, {0, 1}, OnnxPredictionFiltering3d::NormalizationScope::GLOBAL, {128, 128, 128}, 16, OnnxPredictionFiltering3d::OutputNormalizationType::NONE, OnnxPredictionFiltering3d::OutputType::SAME_AS_INPUT );

std::cout << "outputImage:" << result->toString();

chocolate_bar = imagedev.read_vip_image(imagedev_data.get_image_path("chocolate_bar.vip"))

result = imagedev.onnx_prediction_filtering_3d(chocolate_bar, imagedev_data.get_object_path("segmentation3d.onnx"), imagedev.OnnxPredictionFiltering3d.NDHWC, imagedev.OnnxPredictionFiltering3d.InputNormalizationType.NONE, [0, 1], imagedev.OnnxPredictionFiltering3d.GLOBAL, [128, 128, 128], 16, imagedev.OnnxPredictionFiltering3d.OutputNormalizationType.NONE, imagedev.OnnxPredictionFiltering3d.SAME_AS_INPUT)

print("output_image:", str(result))

ImageView chocolate_bar = Data.ReadVipImage( @"Data/images/chocolate_bar.vip" );

IOLink.ImageView result = Processing.OnnxPredictionFiltering3d( chocolate_bar, @"Data/objects/segmentation3d.onnx", OnnxPredictionFiltering3d.DataFormat.NDHWC, OnnxPredictionFiltering3d.InputNormalizationType.NONE, new double[]{0, 1}, OnnxPredictionFiltering3d.NormalizationScope.GLOBAL, new uint[]{128, 128, 128}, 16, OnnxPredictionFiltering3d.OutputNormalizationType.NONE, OnnxPredictionFiltering3d.OutputType.SAME_AS_INPUT );

Console.WriteLine( "outputImage:" + result.ToString() );