StructureEnhancementFilter3d
Enhances structures of interest in a three-dimensional image using a multi-scale analysis.
Access to parameter description
It computes a score between 0 and 1 for each voxel: 1 representing a good match with a structure model and 0 a background pixel. This provides a powerful technique for automatically identifying structures such as blood vessels.
The score can be computed either on an Hessian matrix to detect ridge structures or on a gradient structure tensor for object edges and corners.
The structure models available are:
Where $w(r)$ is a gaussian kernel controlling the scale of analysis.
Let first introduce $ \mathcal{R}_b = \frac{ \lambda_1 }{ \sqrt{ \lambda_2 \lambda_3 } } $, and $ \mathcal{R}_a = \frac{ \lambda_2 }{ \lambda_3 } $
The first ratio attains its maximum for blob-like structure. The second ratio distinguishes plate-like and rod-like structures.
The score for the rod structure type corresponds to:
$$ \text{score}_{\text{ROD}} = (1-\exp(-\frac{\mathcal{R}_a^2}{2\alpha^2}))\exp(-\frac{\mathcal{R}_b^2} {2\beta^2})(1-\exp(-\frac{S^2}{2c^2})) $$ The score for the ball structure type favors isotropic tensors and is computed as follows:
$$ \text{score}_{\text{BALL}} = (1-\exp(-\frac{\mathcal{R}_a^2}{2\alpha^2})) (1-\exp(-\frac{\mathcal{R}_b^2}{2\beta^2}))(1-\exp(-\frac{S^2}{2c^2})) \\ $$ The score for the plane structure type favors isotropic tensors and is computed as follows: $$ \text{score}_{\text{PLANE}} = \exp(-\frac{\mathcal{R}_a^2}{2\alpha^2})\exp(-\frac{\mathcal{R}_b^2} {2\beta^2})(1-\exp(-\frac{S^2}{2c^2})) $$ Where:
The following table summarizes enhanced features according to the parameters. \begin{array}{|l||l|l|l|l|l|} \hline \textbf{Tensor Type} &\textbf{Structure Type} & \textbf{$\lambda_1$} & \textbf{$\lambda_2$} & \textbf{$\lambda_3$} &\textbf{Highlighted features} \\ \hline \hline \text{HESSIAN} & \text{PLANE} & \text{Strong} & \text{Weak} & \text{Weak} & \text{Plate-like structures}\\ \hline \text{HESSIAN} & \text{ROD} & \text{Strong} & \text{Strong} & \text{Weak} & \text{Ridges, tubular structures} \\ \hline \text{HESSIAN} & \text{BALL} & \text{Strong} & \text{Strong} & \text{Strong} & \text{Blobs, ball-like structures}\\ \hline \text{GRADIENT} & \text{PLANE} & \text{Strong} & \text{Weak} & \text{Weak} & \text{Object faces} \\ \hline \text{GRADIENT} & \text{ROD} & \text{Strong} & \text{Strong} & \text{Weak} & \text{Object edges} \\ \hline \text{GRADIENT} & \text{BALL} & \text{Strong} & \text{Strong} & \text{Strong} & \text{Blobs/Object corners} \\ \hline \end{array} See also
Access to parameter description
Overview
This algorithm generates a score image that can be used with the goal of segmenting the input image.It computes a score between 0 and 1 for each voxel: 1 representing a good match with a structure model and 0 a background pixel. This provides a powerful technique for automatically identifying structures such as blood vessels.
The score can be computed either on an Hessian matrix to detect ridge structures or on a gradient structure tensor for object edges and corners.
The structure models available are:
- Balls (spherical structures with Hessian in 2D, object corners with gradient tensor)
- Rods (linear structures)
- Plates (planar structures)
- A set of tensor fields is extracted from the image by filtering the image at different scales.
- A score image is extracted from each tensor field by analyzing the eigenvalues of tensors.
- The final score is obtained by selecting the maximum of all score images.
- A.F. Frangi, W.J. Niessen, K.L. Vincken, M.A.Viergever, "Multiscale vessel enhancement filtering", Lecture Notes in Computer Science (MICCAI), vol.1496, pp. 130-137, 1998.
- A. Martinez-Sanchez, I. Garcia, S. Asano, V. Lucic, and J.-J. Fernandez, "Robust membrane detection based on tensor voting for electron tomography", Journal of Structural Biology, vol.186, issue.1, pp.49-61, 2014.
Tensor Extraction
Two modes are available for computing the tensor field: the gradient mode and the Hessian mode. The first one is based on the gradient structure tensor, the second one on the Hessian matrix.Gradient mode
With the gradient mode, the gradient tensor is extracted. This tensor, also referred to as structure tensor or second order moment matrix, is a matrix derived from the gradient of the image. This matrix summarizes the predominant directions of the gradient around the pixel of interest. $$ S_w(p) = \left( \begin{array}{ccc} \int w(r) (I_x(p - r)) ^ 2\, d r & \int w(r) I_x(p - r)I_y(p - r)\, d r & \int w(r) I_x(p - r)I_z(p - r)\, d r\\ \int w(r) I_x(p - r)I_y(p - r)\, d r & \int w(r) (I_y(p - r)) ^ 2\, d r & \int w(r) I_y(p - r)I_z(p - r)\, d r\\ \int w(r) I_x(p - r)I_z(p - r)\, d r & \int w(r) I_y(p - r)I_z(p - r)\, d r & \int w(r) (I_z(p - r)) ^ 2\, d r\\ \end{array} \right) $$Where $w(r)$ is a gaussian kernel controlling the scale of analysis.
Hessian mode
With this mode, the tensor field is based on the extraction of the Hessian matrix of the image. This Hessian matrix is computed by filtering the image with the derivative of a Gaussian kernel. The standard deviation of the kernel controls the scale of analysis.Feature extraction
The structureType parameter controls the type of structure that is extracted from the tensor field (rod or ball). Let $\lambda_{1}$, $\lambda_{2}$ and $\lambda_{3}$ where $\mid \lambda_{1} \mid \geq \mid \lambda_{2} \mid \geq \mid \lambda_{3} \mid $.Let first introduce $ \mathcal{R}_b = \frac{ \lambda_1 }{ \sqrt{ \lambda_2 \lambda_3 } } $, and $ \mathcal{R}_a = \frac{ \lambda_2 }{ \lambda_3 } $
The first ratio attains its maximum for blob-like structure. The second ratio distinguishes plate-like and rod-like structures.
The score for the rod structure type corresponds to:
$$ \text{score}_{\text{ROD}} = (1-\exp(-\frac{\mathcal{R}_a^2}{2\alpha^2}))\exp(-\frac{\mathcal{R}_b^2} {2\beta^2})(1-\exp(-\frac{S^2}{2c^2})) $$ The score for the ball structure type favors isotropic tensors and is computed as follows:
$$ \text{score}_{\text{BALL}} = (1-\exp(-\frac{\mathcal{R}_a^2}{2\alpha^2})) (1-\exp(-\frac{\mathcal{R}_b^2}{2\beta^2}))(1-\exp(-\frac{S^2}{2c^2})) \\ $$ The score for the plane structure type favors isotropic tensors and is computed as follows: $$ \text{score}_{\text{PLANE}} = \exp(-\frac{\mathcal{R}_a^2}{2\alpha^2})\exp(-\frac{\mathcal{R}_b^2} {2\beta^2})(1-\exp(-\frac{S^2}{2c^2})) $$ Where:
- $S$ is the tensor norm $ S = \sqrt{\sum \lambda_i^2} $,
- $\alpha$ is a threshold which controls the flatness sensitivity,
- $\beta$ is a threshold which controls the blobness sensitivity,
- and $c$ is a sensitivity threshold which controls the noise influence. $c = \text{ noiseCutoff } \times S_{ max }$ with $S_{ max }$ the maximum Hessian norm in the image.
The following table summarizes enhanced features according to the parameters. \begin{array}{|l||l|l|l|l|l|} \hline \textbf{Tensor Type} &\textbf{Structure Type} & \textbf{$\lambda_1$} & \textbf{$\lambda_2$} & \textbf{$\lambda_3$} &\textbf{Highlighted features} \\ \hline \hline \text{HESSIAN} & \text{PLANE} & \text{Strong} & \text{Weak} & \text{Weak} & \text{Plate-like structures}\\ \hline \text{HESSIAN} & \text{ROD} & \text{Strong} & \text{Strong} & \text{Weak} & \text{Ridges, tubular structures} \\ \hline \text{HESSIAN} & \text{BALL} & \text{Strong} & \text{Strong} & \text{Strong} & \text{Blobs, ball-like structures}\\ \hline \text{GRADIENT} & \text{PLANE} & \text{Strong} & \text{Weak} & \text{Weak} & \text{Object faces} \\ \hline \text{GRADIENT} & \text{ROD} & \text{Strong} & \text{Strong} & \text{Weak} & \text{Object edges} \\ \hline \text{GRADIENT} & \text{BALL} & \text{Strong} & \text{Strong} & \text{Strong} & \text{Blobs/Object corners} \\ \hline \end{array} See also
Function Syntax
This function returns outputImage.
// Function prototype
std::shared_ptr< iolink::ImageView > structureEnhancementFilter3d( std::shared_ptr< iolink::ImageView > inputImage, StructureEnhancementFilter3d::TensorType tensorType, iolink::Vector2d standardDeviationRange, double standardDeviationStep, StructureEnhancementFilter3d::Lightness lightness, StructureEnhancementFilter3d::StructureType structureType, std::shared_ptr< iolink::ImageView > outputImage = nullptr );
This function returns outputImage.
// Function prototype. structure_enhancement_filter_3d(input_image: idt.ImageType, tensor_type: StructureEnhancementFilter3d.TensorType = StructureEnhancementFilter3d.TensorType.HESSIAN, standard_deviation_range: Union[Iterable[int], Iterable[float]] = [1, 3], standard_deviation_step: float = 1, lightness: StructureEnhancementFilter3d.Lightness = StructureEnhancementFilter3d.Lightness.BRIGHT, structure_type: StructureEnhancementFilter3d.StructureType = StructureEnhancementFilter3d.StructureType.ROD, output_image: idt.ImageType = None) -> idt.ImageType
This function returns outputImage.
// Function prototype. public static IOLink.ImageView StructureEnhancementFilter3d( IOLink.ImageView inputImage, StructureEnhancementFilter3d.TensorType tensorType = ImageDev.StructureEnhancementFilter3d.TensorType.HESSIAN, double[] standardDeviationRange = null, double standardDeviationStep = 1, StructureEnhancementFilter3d.Lightness lightness = ImageDev.StructureEnhancementFilter3d.Lightness.BRIGHT, StructureEnhancementFilter3d.StructureType structureType = ImageDev.StructureEnhancementFilter3d.StructureType.ROD, IOLink.ImageView outputImage = null );
Class Syntax
Parameters
Parameter Name | Description | Type | Supported Values | Default Value | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
inputImage |
The input image. The image type can be integer or floating point. | Image | Grayscale | nullptr | |||||||
tensorType |
The type of tensor to compute: gradient tensor or Hessian matrix.
|
Enumeration | HESSIAN | ||||||||
standardDeviationRange |
Standard deviation of the Gaussian kernel at the minimum and maximum scale. | Vector2d | >=0.1 | {1.f, 3.f} | |||||||
standardDeviationStep |
Standard deviation step. Structures will be detected from minimum to maximum standard deviation at a pitch of this value. | Float64 | >0 | 1 | |||||||
lightness |
The lightness type of structures to enhance.
|
Enumeration | BRIGHT | ||||||||
structureType |
The shape of the structures to enhance.
|
Enumeration | ROD | ||||||||
outputImage |
The output image reprensenting the matching score between 0 and 1. Its dimensions are forced to the same values as the input. Its data type is forced to floating point. | Image | nullptr |
Parameter Name | Description | Type | Supported Values | Default Value | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
input_image |
The input image. The image type can be integer or floating point. | image | Grayscale | None | |||||||
tensor_type |
The type of tensor to compute: gradient tensor or Hessian matrix.
|
enumeration | HESSIAN | ||||||||
standard_deviation_range |
Standard deviation of the Gaussian kernel at the minimum and maximum scale. | vector2d | >=0.1 | [1, 3] | |||||||
standard_deviation_step |
Standard deviation step. Structures will be detected from minimum to maximum standard deviation at a pitch of this value. | float64 | >0 | 1 | |||||||
lightness |
The lightness type of structures to enhance.
|
enumeration | BRIGHT | ||||||||
structure_type |
The shape of the structures to enhance.
|
enumeration | ROD | ||||||||
output_image |
The output image reprensenting the matching score between 0 and 1. Its dimensions are forced to the same values as the input. Its data type is forced to floating point. | image | None |
Parameter Name | Description | Type | Supported Values | Default Value | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
inputImage |
The input image. The image type can be integer or floating point. | Image | Grayscale | null | |||||||
tensorType |
The type of tensor to compute: gradient tensor or Hessian matrix.
|
Enumeration | HESSIAN | ||||||||
standardDeviationRange |
Standard deviation of the Gaussian kernel at the minimum and maximum scale. | Vector2d | >=0.1 | {1f, 3f} | |||||||
standardDeviationStep |
Standard deviation step. Structures will be detected from minimum to maximum standard deviation at a pitch of this value. | Float64 | >0 | 1 | |||||||
lightness |
The lightness type of structures to enhance.
|
Enumeration | BRIGHT | ||||||||
structureType |
The shape of the structures to enhance.
|
Enumeration | ROD | ||||||||
outputImage |
The output image reprensenting the matching score between 0 and 1. Its dimensions are forced to the same values as the input. Its data type is forced to floating point. | Image | null |
Object Examples
auto foam = readVipImage( std::string( IMAGEDEVDATA_IMAGES_FOLDER ) + "foam.vip" ); StructureEnhancementFilter3d structureEnhancementFilter3dAlgo; structureEnhancementFilter3dAlgo.setInputImage( foam ); structureEnhancementFilter3dAlgo.setTensorType( StructureEnhancementFilter3d::TensorType::HESSIAN ); structureEnhancementFilter3dAlgo.setStandardDeviationRange( {1.0, 3.0} ); structureEnhancementFilter3dAlgo.setStandardDeviationStep( 1.0 ); structureEnhancementFilter3dAlgo.setLightness( StructureEnhancementFilter3d::Lightness::BRIGHT ); structureEnhancementFilter3dAlgo.setStructureType( StructureEnhancementFilter3d::StructureType::ROD ); structureEnhancementFilter3dAlgo.execute(); std::cout << "outputImage:" << structureEnhancementFilter3dAlgo.outputImage()->toString();
foam = imagedev.read_vip_image(imagedev_data.get_image_path("foam.vip")) structure_enhancement_filter_3d_algo = imagedev.StructureEnhancementFilter3d() structure_enhancement_filter_3d_algo.input_image = foam structure_enhancement_filter_3d_algo.tensor_type = imagedev.StructureEnhancementFilter3d.HESSIAN structure_enhancement_filter_3d_algo.standard_deviation_range = [1.0, 3.0] structure_enhancement_filter_3d_algo.standard_deviation_step = 1.0 structure_enhancement_filter_3d_algo.lightness = imagedev.StructureEnhancementFilter3d.BRIGHT structure_enhancement_filter_3d_algo.structure_type = imagedev.StructureEnhancementFilter3d.ROD structure_enhancement_filter_3d_algo.execute() print("output_image:", str(structure_enhancement_filter_3d_algo.output_image))
ImageView foam = Data.ReadVipImage( @"Data/images/foam.vip" ); StructureEnhancementFilter3d structureEnhancementFilter3dAlgo = new StructureEnhancementFilter3d { inputImage = foam, tensorType = StructureEnhancementFilter3d.TensorType.HESSIAN, standardDeviationRange = new double[]{1.0, 3.0}, standardDeviationStep = 1.0, lightness = StructureEnhancementFilter3d.Lightness.BRIGHT, structureType = StructureEnhancementFilter3d.StructureType.ROD }; structureEnhancementFilter3dAlgo.Execute(); Console.WriteLine( "outputImage:" + structureEnhancementFilter3dAlgo.outputImage.ToString() );
Function Examples
auto foam = readVipImage( std::string( IMAGEDEVDATA_IMAGES_FOLDER ) + "foam.vip" ); auto result = structureEnhancementFilter3d( foam, StructureEnhancementFilter3d::TensorType::HESSIAN, {1.0, 3.0}, 1.0, StructureEnhancementFilter3d::Lightness::BRIGHT, StructureEnhancementFilter3d::StructureType::ROD ); std::cout << "outputImage:" << result->toString();
foam = imagedev.read_vip_image(imagedev_data.get_image_path("foam.vip")) result = imagedev.structure_enhancement_filter_3d(foam, imagedev.StructureEnhancementFilter3d.HESSIAN, [1.0, 3.0], 1.0, imagedev.StructureEnhancementFilter3d.BRIGHT, imagedev.StructureEnhancementFilter3d.ROD) print("output_image:", str(result))
ImageView foam = Data.ReadVipImage( @"Data/images/foam.vip" ); IOLink.ImageView result = Processing.StructureEnhancementFilter3d( foam, StructureEnhancementFilter3d.TensorType.HESSIAN, new double[]{1.0, 3.0}, 1.0, StructureEnhancementFilter3d.Lightness.BRIGHT, StructureEnhancementFilter3d.StructureType.ROD ); Console.WriteLine( "outputImage:" + result.ToString() );