ImageDev

ExtractSegmentedObjects

Generates a sequence of object-centered image thumbnails and corresponding binary masks from an input image and a label image.

Access to parameter description

This command is experimental, his signature may be modified between now and his final version.

 This algorithm extracts individual objects from an input grayscale or color image using a label image that encodes object instances.

For each unique label value, the operator computes the object’s axis-aligned bounding box, extracts the corresponding region from both the input image and the label image, and produces: The output thumbnails are returned as an image sequence (stack), where each frame represents one object instance.
This operator is typically used to prepare object-centric image data for downstream image-based classification or feature extraction workflows.
Although commonly used after label analysis or segmentation, it is not restricted to any specific labeling workflow and operates on any valid label image.

The output table is a dataframe containing the following columns:
$$ \begin{array}{|c||c|c|} \hline \textbf{Column Name} & \textbf{Type} & \textbf{Description} \\ \hline \mbox{labelValue} & \mbox{UINT64} & \mbox{The label value of the object in the input label image.} \\ \hline \mbox{BoundingBoxOriginX} & \mbox{DOUBLE} & \mbox{The origin X-coordinate of object bounding box in the input image, in pixels.} \\ \hline \mbox{BoundingBoxOriginY} & \mbox{DOUBLE} & \mbox{The origin Y-coordinate of object bounding box in the input image, in pixels.} \\ \hline \mbox{BoundingBoxSizeX} & \mbox{DOUBLE} & \mbox{The size in the X direction of the object bounding box in the input image, in pixels.} \\ \hline \mbox{BoundingBoxSizeY} & \mbox{DOUBLE} & \mbox{The size in the Y direction of the object bounding box in the input image, in pixels.} \\ \hline \mbox{PixelSizeX} & \mbox{DOUBLE} & \mbox{The pixel size in the X direction of the extracted image.} \\ \hline \mbox{PixelSizeY} & \mbox{DOUBLE} & \mbox{The pixel size in the X direction of the extracted image.} \\ \hline \end{array} $$

See also

Function Syntax

This function returns a ExtractSegmentedObjectsOutput structure containing outputSequence, outputBinarySequence and outputTable. 
// Output structure of the extractSegmentedObjects function.
struct ExtractSegmentedObjectsOutput final
{
    /// The stack of resized object thumbnails, containing one image per object instance. It has same type and interpretation as the input image and width and height are defined by imageShape.
    std::shared_ptr< iolink::ImageView > outputSequence;
    /// The stack of resized binary masks derived from the label image. Each frame represents only the corresponding object. Pixel interpretation is binary and width and height are defined by imageShape.
    std::shared_ptr< iolink::ImageView > outputBinarySequence;
    /// The table mapping each sequence index to the corresponding label value in the input label image.
    std::shared_ptr< iolink::DataFrameView > outputTable;
};

// Function prototype
ExtractSegmentedObjectsOutput
extractSegmentedObjects( std::shared_ptr< iolink::ImageView > inputImage,
                         std::shared_ptr< iolink::ImageView > inputLabelImage,
                         ExtractSegmentedObjects::Mode mode,
                         const iolink::Vector2u32& imageShape,
                         const iolink::Vector2d& physicalSize,
                         double paddingValue,
                         std::shared_ptr< iolink::ImageView > outputSequence = nullptr,
                         std::shared_ptr< iolink::ImageView > outputBinarySequence = nullptr,
                         std::shared_ptr< iolink::DataFrameView > outputTable = nullptr );
This function returns a tuple containing output_sequence, output_binary_sequence and output_table. 
// Function prototype.
extract_segmented_objects(input_image: idt.ImageType,
                          input_label_image: idt.ImageType,
                          mode: Union[Literal["FIXED"],Literal["ADJUSTED"],Literal["ADJUSTED_WITH_RATIO"],Literal["FIXED_OR_ADJUSTED_WITH_RATIO"],ExtractSegmentedObjects.Mode] = ExtractSegmentedObjects.Mode.FIXED,
                          image_shape: Iterable[int] = [128, 128],
                          physical_size: Union[Iterable[int], Iterable[float]] = [0, 0],
                          padding_value: float = 0,
                          output_sequence: idt.ImageType = None,
                          output_binary_sequence: idt.ImageType = None,
                          output_table: Union[iolink.DataFrameView, None] = None) -> Tuple[idt.ImageType, idt.ImageType, Union[iolink.DataFrameView, None]]
This function returns a ExtractSegmentedObjectsOutput structure containing outputSequence, outputBinarySequence and outputTable. 
/// Output structure of the ExtractSegmentedObjects function.
public struct ExtractSegmentedObjectsOutput
{
    /// 
    /// The stack of resized object thumbnails, containing one image per object instance. It has same type and interpretation as the input image and width and height are defined by imageShape.
    /// 
    public IOLink.ImageView outputSequence;
    /// 
    /// The stack of resized binary masks derived from the label image. Each frame represents only the corresponding object. Pixel interpretation is binary and width and height are defined by imageShape.
    /// 
    public IOLink.ImageView outputBinarySequence;
    /// 
    /// The table mapping each sequence index to the corresponding label value in the input label image.
    /// 
    public IOLink.DataFrameView outputTable;
};

// Function prototype.
public static ExtractSegmentedObjectsOutput
ExtractSegmentedObjects( IOLink.ImageView inputImage,
                         IOLink.ImageView inputLabelImage,
                         ExtractSegmentedObjects.Mode mode = ImageDev.ExtractSegmentedObjects.Mode.FIXED,
                         uint[] imageShape = null,
                         double[] physicalSize = null,
                         double paddingValue = 0,
                         IOLink.ImageView outputSequence = null,
                         IOLink.ImageView outputBinarySequence = null,
                         IOLink.DataFrameView outputTable = null );

Class Syntax

Parameters

Parameter Name Description Type Supported Values Default Value
input
inputImage
The source image from which object thumbnails are extracted. Image sequences are not allowed. Image Binary, Label, Grayscale or Multispectral nullptr
input
inputLabelImage
The label image where each object instance is represented by a unique, non-zero label value. It should have same physical dimensions as the first input image. Image Binary or Label nullptr
input
imageShape
The desired width and height of the output thumbnails, in pixels. Vector2u32 >0 {128, 128}
input
physicalSize
The desired width and height of the part of the inputImage we want to sample, in physical unit. In case of default value, it is computed as the physical size of the imageShape in the inputImage. Vector2d >=0 {0.f, 0.f}
input
mode
Defines how extracted object regions are adapted to the output size.
Value Description
FIXED Extract regions centered on each object with a physical size of physicalSize. Output patches have shape imageShape, using padding where required.
ADJUSTED Extract regions centered on each object. The physical size is adjusted independently along the two axes to fit the object s shape. The output shape is imageShape. Note that the aspect ratio is not preserved.
ADJUSTED_WITH_RATIO Extract regions centered on each object. The physical size is adjusted along only one axis to fit the object s shape while preserving the aspect ratio. The output shape is imageShape.
FIXED_OR_ADJUSTED_WITH_RATIO Use the FIXED mode if the object can be completly contained in the region of size physicalSize. Use the ADJUSTED_WITH_RATIO mode, otherwise.
Enumeration FIXED
input
paddingValue
The padding value used when information outside the input image is required (FIXED, FIXED_OR_ADJUSTED_WITH_RATIO). Float64 Any value 0
output
outputSequence
The stack of resized object thumbnails, containing one image per object instance. It has same type and interpretation as the input image and width and height are defined by imageShape. Image nullptr
output
outputBinarySequence
The stack of resized binary masks derived from the label image. Each frame represents only the corresponding object. Pixel interpretation is binary and width and height are defined by imageShape. Image nullptr
output
outputTable
The table mapping each sequence index to the corresponding label value in the input label image. DataFrameView nullptr
Parameter Name Description Type Supported Values Default Value
input
input_image
The source image from which object thumbnails are extracted. Image sequences are not allowed. image Binary, Label, Grayscale or Multispectral None
input
input_label_image
The label image where each object instance is represented by a unique, non-zero label value. It should have same physical dimensions as the first input image. image Binary or Label None
input
image_shape
The desired width and height of the output thumbnails, in pixels. vector2u32 >0 [128, 128]
input
physical_size
The desired width and height of the part of the inputImage we want to sample, in physical unit. In case of default value, it is computed as the physical size of the imageShape in the inputImage. vector2d >=0 [0, 0]
input
mode
Defines how extracted object regions are adapted to the output size.
Value Description
FIXED Extract regions centered on each object with a physical size of physicalSize. Output patches have shape imageShape, using padding where required.
ADJUSTED Extract regions centered on each object. The physical size is adjusted independently along the two axes to fit the object s shape. The output shape is imageShape. Note that the aspect ratio is not preserved.
ADJUSTED_WITH_RATIO Extract regions centered on each object. The physical size is adjusted along only one axis to fit the object s shape while preserving the aspect ratio. The output shape is imageShape.
FIXED_OR_ADJUSTED_WITH_RATIO Use the FIXED mode if the object can be completly contained in the region of size physicalSize. Use the ADJUSTED_WITH_RATIO mode, otherwise.
enumeration FIXED
input
padding_value
The padding value used when information outside the input image is required (FIXED, FIXED_OR_ADJUSTED_WITH_RATIO). float64 Any value 0
output
output_sequence
The stack of resized object thumbnails, containing one image per object instance. It has same type and interpretation as the input image and width and height are defined by imageShape. image None
output
output_binary_sequence
The stack of resized binary masks derived from the label image. Each frame represents only the corresponding object. Pixel interpretation is binary and width and height are defined by imageShape. image None
output
output_table
The table mapping each sequence index to the corresponding label value in the input label image. data_frame_view None
Parameter Name Description Type Supported Values Default Value
input
inputImage
The source image from which object thumbnails are extracted. Image sequences are not allowed. Image Binary, Label, Grayscale or Multispectral null
input
inputLabelImage
The label image where each object instance is represented by a unique, non-zero label value. It should have same physical dimensions as the first input image. Image Binary or Label null
input
imageShape
The desired width and height of the output thumbnails, in pixels. Vector2u32 >0 {128, 128}
input
physicalSize
The desired width and height of the part of the inputImage we want to sample, in physical unit. In case of default value, it is computed as the physical size of the imageShape in the inputImage. Vector2d >=0 {0f, 0f}
input
mode
Defines how extracted object regions are adapted to the output size.
Value Description
FIXED Extract regions centered on each object with a physical size of physicalSize. Output patches have shape imageShape, using padding where required.
ADJUSTED Extract regions centered on each object. The physical size is adjusted independently along the two axes to fit the object s shape. The output shape is imageShape. Note that the aspect ratio is not preserved.
ADJUSTED_WITH_RATIO Extract regions centered on each object. The physical size is adjusted along only one axis to fit the object s shape while preserving the aspect ratio. The output shape is imageShape.
FIXED_OR_ADJUSTED_WITH_RATIO Use the FIXED mode if the object can be completly contained in the region of size physicalSize. Use the ADJUSTED_WITH_RATIO mode, otherwise.
Enumeration FIXED
input
paddingValue
The padding value used when information outside the input image is required (FIXED, FIXED_OR_ADJUSTED_WITH_RATIO). Float64 Any value 0
output
outputSequence
The stack of resized object thumbnails, containing one image per object instance. It has same type and interpretation as the input image and width and height are defined by imageShape. Image null
output
outputBinarySequence
The stack of resized binary masks derived from the label image. Each frame represents only the corresponding object. Pixel interpretation is binary and width and height are defined by imageShape. Image null
output
outputTable
The table mapping each sequence index to the corresponding label value in the input label image. DataFrameView null

Object Examples

auto polystyrene = ioformat::readImage( std::string( IMAGEDEVDATA_IMAGES_FOLDER ) + "polystyrene.tif" );
auto polystyrene_sep_label = readVipImage( std::string( IMAGEDEVDATA_IMAGES_FOLDER ) + "polystyrene_sep_label.vip" );

ExtractSegmentedObjects extractSegmentedObjectsAlgo;
extractSegmentedObjectsAlgo.setInputImage( polystyrene );
extractSegmentedObjectsAlgo.setInputLabelImage( polystyrene_sep_label );
extractSegmentedObjectsAlgo.setMode( ExtractSegmentedObjects::Mode::FIXED_OR_ADJUSTED_WITH_RATIO );
extractSegmentedObjectsAlgo.setImageShape( {64, 64} );
extractSegmentedObjectsAlgo.setPhysicalSize( {0, 0} );
extractSegmentedObjectsAlgo.setPaddingValue( 2.0 );
extractSegmentedObjectsAlgo.execute();

std::cout << "outputSequence:" << extractSegmentedObjectsAlgo.outputSequence()->toString();
std::cout << "outputBinarySequence:" << extractSegmentedObjectsAlgo.outputBinarySequence()->toString();
std::cout << "outputTable:" << extractSegmentedObjectsAlgo.outputTable()->shape();

polystyrene = ioformat.read_image(imagedev_data.get_image_path("polystyrene.tif"))
polystyrene_sep_label = imagedev.read_vip_image(imagedev_data.get_image_path("polystyrene_sep_label.vip"))

extract_segmented_objects_algo = imagedev.ExtractSegmentedObjects()
extract_segmented_objects_algo.input_image = polystyrene
extract_segmented_objects_algo.input_label_image = polystyrene_sep_label
extract_segmented_objects_algo.mode = imagedev.ExtractSegmentedObjects.FIXED_OR_ADJUSTED_WITH_RATIO
extract_segmented_objects_algo.image_shape = [64, 64]
extract_segmented_objects_algo.physical_size = [0, 0]
extract_segmented_objects_algo.padding_value = 2.0
extract_segmented_objects_algo.execute()

print("output_sequence:", str(extract_segmented_objects_algo.output_sequence))
print("output_binary_sequence:", str(extract_segmented_objects_algo.output_binary_sequence))
print("output_table:", str(extract_segmented_objects_algo.output_table))

ImageView polystyrene = ViewIO.ReadImage( @"Data/images/polystyrene.tif" );
ImageView polystyrene_sep_label = Data.ReadVipImage( @"Data/images/polystyrene_sep_label.vip" );

ExtractSegmentedObjects extractSegmentedObjectsAlgo = new ExtractSegmentedObjects
{
    inputImage = polystyrene,
    inputLabelImage = polystyrene_sep_label,
    mode = ExtractSegmentedObjects.Mode.FIXED_OR_ADJUSTED_WITH_RATIO,
    imageShape = new uint[]{64, 64},
    physicalSize = new double[]{0, 0},
    paddingValue = 2.0
};
extractSegmentedObjectsAlgo.Execute();

Console.WriteLine( "outputSequence:" + extractSegmentedObjectsAlgo.outputSequence.ToString() );
Console.WriteLine( "outputBinarySequence:" + extractSegmentedObjectsAlgo.outputBinarySequence.ToString() );
Console.WriteLine( "outputTable:" + extractSegmentedObjectsAlgo.outputTable.ToString() );

Function Examples

auto polystyrene = ioformat::readImage( std::string( IMAGEDEVDATA_IMAGES_FOLDER ) + "polystyrene.tif" );
auto polystyrene_sep_label = readVipImage( std::string( IMAGEDEVDATA_IMAGES_FOLDER ) + "polystyrene_sep_label.vip" );

auto result = extractSegmentedObjects( polystyrene, polystyrene_sep_label, ExtractSegmentedObjects::Mode::FIXED_OR_ADJUSTED_WITH_RATIO, {64, 64}, {0, 0}, 2.0 );

std::cout << "outputSequence:" << result.outputSequence->toString();
std::cout << "outputBinarySequence:" << result.outputBinarySequence->toString();
std::cout << "outputTable:" << result.outputTable->shape();

polystyrene = ioformat.read_image(imagedev_data.get_image_path("polystyrene.tif"))
polystyrene_sep_label = imagedev.read_vip_image(imagedev_data.get_image_path("polystyrene_sep_label.vip"))

result_output_sequence, result_output_binary_sequence, result_output_table = imagedev.extract_segmented_objects(polystyrene, polystyrene_sep_label, imagedev.ExtractSegmentedObjects.FIXED_OR_ADJUSTED_WITH_RATIO, [64, 64], [0, 0], 2.0)

print("output_sequence:", str(result_output_sequence))
print("output_binary_sequence:", str(result_output_binary_sequence))
print("output_table:", str(result_output_table))

ImageView polystyrene = ViewIO.ReadImage( @"Data/images/polystyrene.tif" );
ImageView polystyrene_sep_label = Data.ReadVipImage( @"Data/images/polystyrene_sep_label.vip" );

Processing.ExtractSegmentedObjectsOutput result = Processing.ExtractSegmentedObjects( polystyrene, polystyrene_sep_label, ExtractSegmentedObjects.Mode.FIXED_OR_ADJUSTED_WITH_RATIO, new uint[]{64, 64}, new double[]{0, 0}, 2.0 );

Console.WriteLine( "outputSequence:" + result.outputSequence.ToString() );
Console.WriteLine( "outputBinarySequence:" + result.outputBinarySequence.ToString() );
Console.WriteLine( "outputTable:" + result.outputTable.ToString() );



© 2026 Thermo Fisher Scientific Inc. All rights reserved.