ImageDev

StructureEnhancementFilter3d

Enhances structures of interest in a three-dimensional image using a multi-scale analysis.

Access to parameter description

Overview

This algorithm generates a score image that can be used with the goal of segmenting the input image.

It computes a score between 0 and 1 for each voxel: 1 representing a good match with a structure model and 0 a background pixel. This provides a powerful technique for automatically identifying structures such as blood vessels.
The score can be computed either on an Hessian matrix to detect ridge structures or on a gradient structure tensor for object edges and corners.
The structure models available are: The principle of the algorithm can be summarized as follows: References:

Tensor Extraction

Two modes are available for computing the tensor field: the gradient mode and the Hessian mode. The first one is based on the gradient structure tensor, the second one on the Hessian matrix.

Gradient mode

With the gradient mode, the gradient tensor is extracted. This tensor, also referred to as structure tensor or second order moment matrix, is a matrix derived from the gradient of the image. This matrix summarizes the predominant directions of the gradient around the pixel of interest. $$ S_w(p) = \left( \begin{array}{ccc} \int w(r) (I_x(p - r)) ^ 2\, d r & \int w(r) I_x(p - r)I_y(p - r)\, d r & \int w(r) I_x(p - r)I_z(p - r)\, d r\\ \int w(r) I_x(p - r)I_y(p - r)\, d r & \int w(r) (I_y(p - r)) ^ 2\, d r & \int w(r) I_y(p - r)I_z(p - r)\, d r\\ \int w(r) I_x(p - r)I_z(p - r)\, d r & \int w(r) I_y(p - r)I_z(p - r)\, d r & \int w(r) (I_z(p - r)) ^ 2\, d r\\ \end{array} \right) $$
Where $w(r)$ is a gaussian kernel controlling the scale of analysis.

Hessian mode

With this mode, the tensor field is based on the extraction of the Hessian matrix of the image. This Hessian matrix is computed by filtering the image with the derivative of a Gaussian kernel. The standard deviation of the kernel controls the scale of analysis.

Feature extraction

The structureType parameter controls the type of structure that is extracted from the tensor field (rod or ball). Let $\lambda_{1}$, $\lambda_{2}$ and $\lambda_{3}$ where $\mid \lambda_{1} \mid \geq \mid \lambda_{2} \mid \geq \mid \lambda_{3} \mid $.

Let first introduce $ \mathcal{R}_b = \frac{ \lambda_1 }{ \sqrt{ \lambda_2 \lambda_3 } } $, and $ \mathcal{R}_a = \frac{ \lambda_2 }{ \lambda_3 } $
The first ratio attains its maximum for blob-like structure. The second ratio distinguishes plate-like and rod-like structures.
The score for the rod structure type corresponds to:
$$ \text{score}_{\text{ROD}} = (1-\exp(-\frac{\mathcal{R}_a^2}{2\alpha^2}))\exp(-\frac{\mathcal{R}_b^2} {2\beta^2})(1-\exp(-\frac{S^2}{2c^2})) $$ The score for the ball structure type favors isotropic tensors and is computed as follows:
$$ \text{score}_{\text{BALL}} = (1-\exp(-\frac{\mathcal{R}_a^2}{2\alpha^2})) (1-\exp(-\frac{\mathcal{R}_b^2}{2\beta^2}))(1-\exp(-\frac{S^2}{2c^2})) \\ $$ The score for the plane structure type favors isotropic tensors and is computed as follows: $$ \text{score}_{\text{PLANE}} = \exp(-\frac{\mathcal{R}_a^2}{2\alpha^2})\exp(-\frac{\mathcal{R}_b^2} {2\beta^2})(1-\exp(-\frac{S^2}{2c^2})) $$ Where: With the Hessian mode, the lightness parameter limits the feature extraction to dark or bright objects by analyzing the sign of eigenvalues. This parameter is ignored in gradient mode where tensors are positive definite matrices.

The following table summarizes enhanced features according to the parameters. \begin{array}{|l||l|l|l|l|l|} \hline \textbf{Tensor Type} &\textbf{Structure Type} & \textbf{$\lambda_1$} & \textbf{$\lambda_2$} & \textbf{$\lambda_3$} &\textbf{Highlighted features} \\ \hline \hline \text{HESSIAN} & \text{PLANE} & \text{Strong} & \text{Weak} & \text{Weak} & \text{Plate-like structures}\\ \hline \text{HESSIAN} & \text{ROD} & \text{Strong} & \text{Strong} & \text{Weak} & \text{Ridges, tubular structures} \\ \hline \text{HESSIAN} & \text{BALL} & \text{Strong} & \text{Strong} & \text{Strong} & \text{Blobs, ball-like structures}\\ \hline \text{GRADIENT} & \text{PLANE} & \text{Strong} & \text{Weak} & \text{Weak} & \text{Object faces} \\ \hline \text{GRADIENT} & \text{ROD} & \text{Strong} & \text{Strong} & \text{Weak} & \text{Object edges} \\ \hline \text{GRADIENT} & \text{BALL} & \text{Strong} & \text{Strong} & \text{Strong} & \text{Blobs/Object corners} \\ \hline \end{array} See also

Function Syntax

This function returns outputImage. 
// Function prototype
std::shared_ptr< iolink::ImageView >
structureEnhancementFilter3d( std::shared_ptr< iolink::ImageView > inputImage,
                              StructureEnhancementFilter3d::TensorType tensorType,
                              iolink::Vector2d standardDeviationRange,
                              double standardDeviationStep,
                              StructureEnhancementFilter3d::Lightness lightness,
                              StructureEnhancementFilter3d::StructureType structureType,
                              std::shared_ptr< iolink::ImageView > outputImage = nullptr );
This function returns outputImage. 
// Function prototype.
structure_enhancement_filter_3d(input_image: idt.ImageType,
                                tensor_type: StructureEnhancementFilter3d.TensorType = StructureEnhancementFilter3d.TensorType.HESSIAN,
                                standard_deviation_range: Union[Iterable[int], Iterable[float]] = [1, 3],
                                standard_deviation_step: float = 1,
                                lightness: StructureEnhancementFilter3d.Lightness = StructureEnhancementFilter3d.Lightness.BRIGHT,
                                structure_type: StructureEnhancementFilter3d.StructureType = StructureEnhancementFilter3d.StructureType.ROD,
                                output_image: idt.ImageType = None) -> idt.ImageType
This function returns outputImage. 
// Function prototype.
public static IOLink.ImageView
StructureEnhancementFilter3d( IOLink.ImageView inputImage,
                              StructureEnhancementFilter3d.TensorType tensorType = ImageDev.StructureEnhancementFilter3d.TensorType.HESSIAN,
                              double[] standardDeviationRange = null,
                              double standardDeviationStep = 1,
                              StructureEnhancementFilter3d.Lightness lightness = ImageDev.StructureEnhancementFilter3d.Lightness.BRIGHT,
                              StructureEnhancementFilter3d.StructureType structureType = ImageDev.StructureEnhancementFilter3d.StructureType.ROD,
                              IOLink.ImageView outputImage = null );

Class Syntax

Parameters

Parameter Name Description Type Supported Values Default Value
input
inputImage
The input image. The image type can be integer or floating point. Image Grayscale nullptr
input
tensorType
The type of tensor to compute: gradient tensor or Hessian matrix.
HESSIAN The Hessian matrix is extracted.
GRADIENT The gradient structure tensor is extracted.
Enumeration HESSIAN
input
standardDeviationRange
Standard deviation of the Gaussian kernel at the minimum and maximum scale. Vector2d >=0.1 {1.f, 3.f}
input
standardDeviationStep
Standard deviation step. Structures will be detected from minimum to maximum standard deviation at a pitch of this value. Float64 >0 1
input
lightness
The lightness type of structures to enhance.
BRIGHT Extracts bright structures from dark background.
DARK Extracts dark structures from bright background.
Enumeration BRIGHT
input
structureType
The shape of the structures to enhance.
ROD The tubular structures are enhanced.
BALL The blob-like structures are enhanced.
PLANE The plate-like structures are enhanced.
Enumeration ROD
output
outputImage
The output image reprensenting the matching score between 0 and 1. Its dimensions are forced to the same values as the input. Its data type is forced to floating point. Image nullptr
Parameter Name Description Type Supported Values Default Value
input
input_image
The input image. The image type can be integer or floating point. image Grayscale None
input
tensor_type
The type of tensor to compute: gradient tensor or Hessian matrix.
HESSIAN The Hessian matrix is extracted.
GRADIENT The gradient structure tensor is extracted.
enumeration HESSIAN
input
standard_deviation_range
Standard deviation of the Gaussian kernel at the minimum and maximum scale. vector2d >=0.1 [1, 3]
input
standard_deviation_step
Standard deviation step. Structures will be detected from minimum to maximum standard deviation at a pitch of this value. float64 >0 1
input
lightness
The lightness type of structures to enhance.
BRIGHT Extracts bright structures from dark background.
DARK Extracts dark structures from bright background.
enumeration BRIGHT
input
structure_type
The shape of the structures to enhance.
ROD The tubular structures are enhanced.
BALL The blob-like structures are enhanced.
PLANE The plate-like structures are enhanced.
enumeration ROD
output
output_image
The output image reprensenting the matching score between 0 and 1. Its dimensions are forced to the same values as the input. Its data type is forced to floating point. image None
Parameter Name Description Type Supported Values Default Value
input
inputImage
The input image. The image type can be integer or floating point. Image Grayscale null
input
tensorType
The type of tensor to compute: gradient tensor or Hessian matrix.
HESSIAN The Hessian matrix is extracted.
GRADIENT The gradient structure tensor is extracted.
Enumeration HESSIAN
input
standardDeviationRange
Standard deviation of the Gaussian kernel at the minimum and maximum scale. Vector2d >=0.1 {1f, 3f}
input
standardDeviationStep
Standard deviation step. Structures will be detected from minimum to maximum standard deviation at a pitch of this value. Float64 >0 1
input
lightness
The lightness type of structures to enhance.
BRIGHT Extracts bright structures from dark background.
DARK Extracts dark structures from bright background.
Enumeration BRIGHT
input
structureType
The shape of the structures to enhance.
ROD The tubular structures are enhanced.
BALL The blob-like structures are enhanced.
PLANE The plate-like structures are enhanced.
Enumeration ROD
output
outputImage
The output image reprensenting the matching score between 0 and 1. Its dimensions are forced to the same values as the input. Its data type is forced to floating point. Image null

Object Examples

auto foam = readVipImage( std::string( IMAGEDEVDATA_IMAGES_FOLDER ) + "foam.vip" );

StructureEnhancementFilter3d structureEnhancementFilter3dAlgo;
structureEnhancementFilter3dAlgo.setInputImage( foam );
structureEnhancementFilter3dAlgo.setTensorType( StructureEnhancementFilter3d::TensorType::HESSIAN );
structureEnhancementFilter3dAlgo.setStandardDeviationRange( {1.0, 3.0} );
structureEnhancementFilter3dAlgo.setStandardDeviationStep( 1.0 );
structureEnhancementFilter3dAlgo.setLightness( StructureEnhancementFilter3d::Lightness::BRIGHT );
structureEnhancementFilter3dAlgo.setStructureType( StructureEnhancementFilter3d::StructureType::ROD );
structureEnhancementFilter3dAlgo.execute();

std::cout << "outputImage:" << structureEnhancementFilter3dAlgo.outputImage()->toString();

foam = imagedev.read_vip_image(imagedev_data.get_image_path("foam.vip"))

structure_enhancement_filter_3d_algo = imagedev.StructureEnhancementFilter3d()
structure_enhancement_filter_3d_algo.input_image = foam
structure_enhancement_filter_3d_algo.tensor_type = imagedev.StructureEnhancementFilter3d.HESSIAN
structure_enhancement_filter_3d_algo.standard_deviation_range = [1.0, 3.0]
structure_enhancement_filter_3d_algo.standard_deviation_step = 1.0
structure_enhancement_filter_3d_algo.lightness = imagedev.StructureEnhancementFilter3d.BRIGHT
structure_enhancement_filter_3d_algo.structure_type = imagedev.StructureEnhancementFilter3d.ROD
structure_enhancement_filter_3d_algo.execute()

print("output_image:", str(structure_enhancement_filter_3d_algo.output_image))

ImageView foam = Data.ReadVipImage( @"Data/images/foam.vip" );

StructureEnhancementFilter3d structureEnhancementFilter3dAlgo = new StructureEnhancementFilter3d
{
    inputImage = foam,
    tensorType = StructureEnhancementFilter3d.TensorType.HESSIAN,
    standardDeviationRange = new double[]{1.0, 3.0},
    standardDeviationStep = 1.0,
    lightness = StructureEnhancementFilter3d.Lightness.BRIGHT,
    structureType = StructureEnhancementFilter3d.StructureType.ROD
};
structureEnhancementFilter3dAlgo.Execute();

Console.WriteLine( "outputImage:" + structureEnhancementFilter3dAlgo.outputImage.ToString() );

Function Examples

auto foam = readVipImage( std::string( IMAGEDEVDATA_IMAGES_FOLDER ) + "foam.vip" );

auto result = structureEnhancementFilter3d( foam, StructureEnhancementFilter3d::TensorType::HESSIAN, {1.0, 3.0}, 1.0, StructureEnhancementFilter3d::Lightness::BRIGHT, StructureEnhancementFilter3d::StructureType::ROD );

std::cout << "outputImage:" << result->toString();

foam = imagedev.read_vip_image(imagedev_data.get_image_path("foam.vip"))

result = imagedev.structure_enhancement_filter_3d(foam, imagedev.StructureEnhancementFilter3d.HESSIAN, [1.0, 3.0], 1.0, imagedev.StructureEnhancementFilter3d.BRIGHT, imagedev.StructureEnhancementFilter3d.ROD)

print("output_image:", str(result))

ImageView foam = Data.ReadVipImage( @"Data/images/foam.vip" );

IOLink.ImageView result = Processing.StructureEnhancementFilter3d( foam, StructureEnhancementFilter3d.TensorType.HESSIAN, new double[]{1.0, 3.0}, 1.0, StructureEnhancementFilter3d.Lightness.BRIGHT, StructureEnhancementFilter3d.StructureType.ROD );

Console.WriteLine( "outputImage:" + result.ToString() );