Loading...
Searching...
No Matches
pcl::LineRGBD< PointXYZT, PointRGBT > Class Template Reference
Module recognition

High-level class for template matching using the LINEMOD approach based on RGB and Depth data. More...

`#include <pcl/recognition/linemod/line_rgbd.h>`

## Classes

struct | Detection
| A LineRGBD detection. More...

## Public Member Functions

| LineRGBD ()
| Constructor.
virtual | ~LineRGBD ()=default
| Destructor.
bool | loadTemplates (const std::string &file_name, std::size_t object_id=0)
| Loads templates from a LMT (LineMod Template) file.
bool | addTemplate (const SparseQuantizedMultiModTemplate &sqmmt, pcl::PointCloud< pcl::PointXYZRGBA > &cloud, std::size_t object_id=0)
void | setDetectionThreshold (float threshold)
| Sets the threshold for the detection responses.
void | setGradientMagnitudeThreshold (const float threshold)
| Sets the threshold on the magnitude of color gradients.
void | setIntersectionVolumeThreshold (const float threshold=1.0f)
| Sets the threshold for the decision whether two detections of the same template are merged or not.
void | setInputCloud (const typename pcl::PointCloud< PointXYZT >::ConstPtr &cloud)
| Sets the input cloud with xyz point coordinates.
void | setInputColors (const typename pcl::PointCloud< PointRGBT >::ConstPtr &cloud)
| Sets the input cloud with rgb values.
int | createAndAddTemplate (pcl::PointCloud< pcl::PointXYZRGBA > &cloud, const std::size_t object_id, const MaskMap &mask_xyz, const MaskMap &mask_rgb, const RegionXY &region)
| Creates a template from the specified data and adds it to the matching queue.
void | detect (std::vector< typename pcl::LineRGBD< PointXYZT, PointRGBT >::Detection > &detections)
| Applies the detection process and fills the supplied vector with the detection instances.
void | detectSemiScaleInvariant (std::vector< typename pcl::LineRGBD< PointXYZT, PointRGBT >::Detection > &detections, float min_scale=0.6944444f, float max_scale=1.44f, float scale_multiplier=1.2f)
| Applies the detection process in a semi-scale-invariant manner.
void | computeTransformedTemplatePoints (const std::size_t detection_id, pcl::PointCloud< pcl::PointXYZRGBA > &cloud)
| Computes and returns the point cloud of the specified detection.
std::vector< std::size_t > | findObjectPointIndices (const std::size_t detection_id)
| Finds the indices of the points in the input cloud which correspond to the specified detection.

## Public Attributes

pcl::LINEMOD | linemod_
| LINEMOD instance.
pcl::ColorGradientModality< PointRGBT > | color_gradient_mod_
| Color gradient modality.
pcl::SurfaceNormalModality< PointXYZT > | surface_normal_mod_
| Surface normal modality.
pcl::PointCloud< PointXYZT >::ConstPtr | cloud_xyz_
| XYZ point cloud.
pcl::PointCloud< PointRGBT >::ConstPtr | cloud_rgb_
| RGB point cloud.
pcl::PointCloud< pcl::PointXYZRGBA >::CloudVectorType | template_point_clouds_
| Point clouds corresponding to the templates.
std::vector< pcl::BoundingBoxXYZ > | bounding_boxes_
| Bounding boxes corresponding to the templates.
std::vector< std::size_t > | object_ids_
| Object IDs corresponding to the templates.
std::vector< typename pcl::LineRGBD< PointXYZT, PointRGBT >::Detection > | detections_
| Detections from last call of method detect (...).

## Protected Member Functions

std::vector< std::size_t > | alignTemplatePoints (const std::size_t detection_id)
| Aligns the template points with the points found at the detection position of the specified detection.
void | refineDetectionsAlongDepth ()
| Refines the found detections along the depth.
void | applyProjectiveDepthICPOnDetections ()
| Applies projective ICP on detections to find their correct position in depth.
void | removeOverlappingDetections ()
| Checks for overlapping detections, removes them and keeps only the strongest one.

## Static Protected Member Functions

static float | computeBoundingBoxIntersectionVolume (const BoundingBoxXYZ &box1, const BoundingBoxXYZ &box2)
| Computes the volume of the intersection between two bounding boxes.

## Detailed Description

template<typename PointXYZT, typename PointRGBT = PointXYZT>
class pcl::LineRGBD< PointXYZT, PointRGBT >

High-level class for template matching using the LINEMOD approach based on RGB and Depth data.

Author
  Stefan Holzer

Definition at line 73 of file line_rgbd.h.

## Constructor & Destructor Documentation

## LineRGBD()

template<typename PointXYZT, typename PointRGBT = PointXYZT>

pcl::LineRGBD< PointXYZT, PointRGBT >::LineRGBD | ( |  | ) |

inline

Constructor.

Definition at line 98 of file line_rgbd.h.

References cloud_rgb_, cloud_xyz_, color_gradient_mod_, detections_, and surface_normal_mod_.

## ~LineRGBD()

template<typename PointXYZT, typename PointRGBT = PointXYZT>

virtual pcl::LineRGBD< PointXYZT, PointRGBT >::~LineRGBD | ( |  | ) |

virtual default

Destructor.

References addTemplate(), and loadTemplates().

## Member Function Documentation

## addTemplate()

template<typename PointXYZT, typename PointRGBT>

bool pcl::LineRGBD< PointXYZT, PointRGBT >::addTemplate | ( | const SparseQuantizedMultiModTemplate & | sqmmt,
|  | pcl::PointCloud< pcl::PointXYZRGBA > & | cloud,
|  | std::size_t | object_id = 0 )

Definition at line 314 of file line_rgbd.hpp.

References bounding_boxes_, pcl::copyPointCloud(), pcl::BoundingBoxXYZ::depth, pcl::BoundingBoxXYZ::height, linemod_, object_ids_, template_point_clouds_, pcl::BoundingBoxXYZ::width, pcl::BoundingBoxXYZ::x, pcl::BoundingBoxXYZ::y, and pcl::BoundingBoxXYZ::z.

Referenced by ~LineRGBD().

## alignTemplatePoints()

template<typename PointXYZT, typename PointRGBT = PointXYZT>

std::vector< std::size_t > pcl::LineRGBD< PointXYZT, PointRGBT >::alignTemplatePoints | ( | const std::size_t | detection_id | ) |

inline protected

Aligns the template points with the points found at the detection position of the specified detection.

The detection ID refers to the last call of the method detect (...).

Parameters

  [in] | detection_id | The ID of the detection (according to the last call of the method detect (...)).

Definition at line 248 of file line_rgbd.h.

References detections_.

## applyProjectiveDepthICPOnDetections()

template<typename PointXYZT, typename PointRGBT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::applyProjectiveDepthICPOnDetections | ( |  | ) |

protected

Applies projective ICP on detections to find their correct position in depth.

Definition at line 721 of file line_rgbd.hpp.

References pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::bounding_box, bounding_boxes_, detections_, pcl::PointCloud< PointT >::height, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::region, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::template_id, template_point_clouds_, pcl::PointCloud< PointT >::width, pcl::RegionXY::x, pcl::RegionXY::y, and pcl::BoundingBoxXYZ::z.

## computeBoundingBoxIntersectionVolume()

template<typename PointXYZT, typename PointRGBT>

float pcl::LineRGBD< PointXYZT, PointRGBT >::computeBoundingBoxIntersectionVolume | ( | const BoundingBoxXYZ & | box1,
|  | const BoundingBoxXYZ & | box2 )

static protected

Computes the volume of the intersection between two bounding boxes.

Parameters

  [in] | box1 | First bounding box.
  [in] | box2 | Second bounding box.

Definition at line 936 of file line_rgbd.hpp.

References pcl::BoundingBoxXYZ::depth, pcl::BoundingBoxXYZ::height, pcl::BoundingBoxXYZ::width, pcl::BoundingBoxXYZ::x, pcl::BoundingBoxXYZ::y, and pcl::BoundingBoxXYZ::z.

Referenced by removeOverlappingDetections().

## computeTransformedTemplatePoints()

template<typename PointXYZT, typename PointRGBT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::computeTransformedTemplatePoints | ( | const std::size_t | detection_id,
|  | pcl::PointCloud< pcl::PointXYZRGBA > & | cloud )

Computes and returns the point cloud of the specified detection.

This is the template point cloud transformed to the detection coordinates. The detection ID refers to the last call of the method detect (...).

Parameters

  [in] | detection_id | The ID of the detection (according to the last call of the method detect (...)).
  [out] | cloud | The storage for the transformed points.

Definition at line 594 of file line_rgbd.hpp.

References bounding_boxes_, detections_, pcl::PointCloud< PointT >::height, pcl::PointCloud< PointT >::resize(), pcl::PointCloud< PointT >::size(), template_point_clouds_, pcl::PointCloud< PointT >::width, pcl::BoundingBoxXYZ::x, pcl::BoundingBoxXYZ::y, and pcl::BoundingBoxXYZ::z.

## createAndAddTemplate()

template<typename PointXYZT, typename PointRGBT>

int pcl::LineRGBD< PointXYZT, PointRGBT >::createAndAddTemplate | ( | pcl::PointCloud< pcl::PointXYZRGBA > & | cloud,
|  | const std::size_t | object_id,
|  | const MaskMap & | mask_xyz,
|  | const MaskMap & | mask_rgb,
|  | const RegionXY & | region )

Creates a template from the specified data and adds it to the matching queue.

Parameters

  | cloud |
  | object_id |
  [in] | mask_xyz | the mask that determine which parts of the xyz-modality are used for creating the template.
  [in] | mask_rgb | the mask that determine which parts of the rgb-modality are used for creating the template.
  [in] | region | the region which will be associated with the template (can be larger than the actual modality-maps).

Definition at line 221 of file line_rgbd.hpp.

References bounding_boxes_, color_gradient_mod_, pcl::copyPointCloud(), pcl::BoundingBoxXYZ::depth, pcl::BoundingBoxXYZ::height, linemod_, object_ids_, surface_normal_mod_, template_point_clouds_, pcl::BoundingBoxXYZ::width, pcl::BoundingBoxXYZ::x, pcl::BoundingBoxXYZ::y, and pcl::BoundingBoxXYZ::z.

## detect()

template<typename PointXYZT, typename PointRGBT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::detect | ( | std::vector< typename pcl::LineRGBD< PointXYZT, PointRGBT >::Detection > & | detections | ) |

Applies the detection process and fills the supplied vector with the detection instances.

Parameters

  [out] | detections | The storage for the detection instances.

Definition at line 395 of file line_rgbd.hpp.

References pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::bounding_box, bounding_boxes_, cloud_xyz_, color_gradient_mod_, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::detection_id, detections_, pcl::RegionXY::height, linemod_, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::object_id, object_ids_, refineDetectionsAlongDepth(), pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::region, pcl::SparseQuantizedMultiModTemplate::region, removeOverlappingDetections(), pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::response, pcl::LINEMODDetection::score, surface_normal_mod_, pcl::LINEMODDetection::template_id, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::template_id, pcl::RegionXY::width, pcl::BoundingBoxXYZ::x, pcl::LINEMODDetection::x, pcl::RegionXY::x, pcl::BoundingBoxXYZ::y, pcl::LINEMODDetection::y, pcl::RegionXY::y, and pcl::BoundingBoxXYZ::z.

## detectSemiScaleInvariant()

template<typename PointXYZT, typename PointRGBT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::detectSemiScaleInvariant | ( | std::vector< typename pcl::LineRGBD< PointXYZT, PointRGBT >::Detection > & | detections,
|  | float | min_scale = 0.6944444f,
|  | float | max_scale = 1.44f,
|  | float | scale_multiplier = 1.2f )

Applies the detection process in a semi-scale-invariant manner.

This is done by actually scaling the template to different sizes.

Definition at line 493 of file line_rgbd.hpp.

References pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::bounding_box, bounding_boxes_, cloud_xyz_, color_gradient_mod_, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::detection_id, detections_, pcl::RegionXY::height, linemod_, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::object_id, object_ids_, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::region, pcl::SparseQuantizedMultiModTemplate::region, removeOverlappingDetections(), pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::response, pcl::LINEMODDetection::scale, pcl::LINEMODDetection::score, surface_normal_mod_, pcl::LINEMODDetection::template_id, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::template_id, pcl::RegionXY::width, pcl::BoundingBoxXYZ::x, pcl::LINEMODDetection::x, pcl::RegionXY::x, pcl::BoundingBoxXYZ::y, pcl::LINEMODDetection::y, pcl::RegionXY::y, and pcl::BoundingBoxXYZ::z.

## findObjectPointIndices()

template<typename PointXYZT, typename PointRGBT = PointXYZT>

std::vector< std::size_t > pcl::LineRGBD< PointXYZT, PointRGBT >::findObjectPointIndices | ( | const std::size_t | detection_id | ) |

inline

Finds the indices of the points in the input cloud which correspond to the specified detection.

The detection ID refers to the last call of the method detect (...).

Parameters

  [in] | detection_id | The ID of the detection (according to the last call of the method detect (...)).

Definition at line 229 of file line_rgbd.h.

References detections_.

## loadTemplates()

template<typename PointXYZT, typename PointRGBT>

bool pcl::LineRGBD< PointXYZT, PointRGBT >::loadTemplates | ( | const std::string & | file_name,
|  | std::size_t | object_id = 0 )

Loads templates from a LMT (LineMod Template) file.

Overrides old templates.

LineMod Template files are TAR files that store pairs of PCD datasets together with their LINEMOD signatures in SparseQuantizedMultiModTemplate format.

Parameters

  [in] | file_name | The name of the file that stores the templates.
  | object_id |

Returns
  true, if the operation was successful, false otherwise.

Definition at line 77 of file line_rgbd.hpp.

References bounding_boxes_, pcl::BoundingBoxXYZ::depth, pcl::SparseQuantizedMultiModTemplate::deserialize(), pcl::io::TARHeader::file_name, pcl::io::TARHeader::getFileSize(), pcl::BoundingBoxXYZ::height, linemod_, object_ids_, pcl::io::raw_close(), pcl::io::raw_lseek(), pcl::io::raw_open(), pcl::io::raw_read(), pcl::PCDReader::read(), template_point_clouds_, pcl::BoundingBoxXYZ::width, pcl::BoundingBoxXYZ::x, pcl::BoundingBoxXYZ::y, and pcl::BoundingBoxXYZ::z.

Referenced by ~LineRGBD().

## refineDetectionsAlongDepth()

template<typename PointXYZT, typename PointRGBT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::refineDetectionsAlongDepth | ( |  | ) |

protected

Refines the found detections along the depth.

Definition at line 641 of file line_rgbd.hpp.

References pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::bounding_box, cloud_xyz_, pcl::BoundingBoxXYZ::depth, detections_, pcl::RegionXY::height, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::region, pcl::RegionXY::width, pcl::RegionXY::x, pcl::RegionXY::y, and pcl::BoundingBoxXYZ::z.

Referenced by detect().

## removeOverlappingDetections()

template<typename PointXYZT, typename PointRGBT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::removeOverlappingDetections | ( |  | ) |

protected

Checks for overlapping detections, removes them and keeps only the strongest one.

Definition at line 802 of file line_rgbd.hpp.

References pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::bounding_box, computeBoundingBoxIntersectionVolume(), pcl::BoundingBoxXYZ::depth, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::detection_id, detections_, pcl::BoundingBoxXYZ::height, pcl::RegionXY::height, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::object_id, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::region, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::response, pcl::LineRGBD< PointXYZT, PointRGBT >::Detection::template_id, pcl::BoundingBoxXYZ::width, pcl::RegionXY::width, pcl::BoundingBoxXYZ::x, pcl::RegionXY::x, pcl::BoundingBoxXYZ::y, pcl::RegionXY::y, and pcl::BoundingBoxXYZ::z.

Referenced by detect(), and detectSemiScaleInvariant().

## setDetectionThreshold()

template<typename PointXYZT, typename PointRGBT = PointXYZT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::setDetectionThreshold | ( | float | threshold | ) |

inline

Sets the threshold for the detection responses.

Responses are between 0 and 1, where 1 is a best.

Parameters

  [in] | threshold | The threshold used to decide where a template is detected.

Definition at line 131 of file line_rgbd.h.

References linemod_.

## setGradientMagnitudeThreshold()

template<typename PointXYZT, typename PointRGBT = PointXYZT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::setGradientMagnitudeThreshold | ( | const float | threshold | ) |

inline

Sets the threshold on the magnitude of color gradients.

Color gradients with a magnitude below this threshold are not considered in the detection process.

Parameters

  [in] | threshold | The threshold on the magnitude of color gradients.

Definition at line 141 of file line_rgbd.h.

References color_gradient_mod_.

## setInputCloud()

template<typename PointXYZT, typename PointRGBT = PointXYZT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::setInputCloud | ( | const typename pcl::PointCloud< PointXYZT >::ConstPtr & | cloud | ) |

inline

Sets the input cloud with xyz point coordinates.

The cloud has to be organized.

Parameters

  [in] | cloud | The input cloud with xyz point coordinates.

Definition at line 163 of file line_rgbd.h.

References cloud_xyz_, and surface_normal_mod_.

## setInputColors()

template<typename PointXYZT, typename PointRGBT = PointXYZT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::setInputColors | ( | const typename pcl::PointCloud< PointRGBT >::ConstPtr & | cloud | ) |

inline

Sets the input cloud with rgb values.

The cloud has to be organized.

Parameters

  [in] | cloud | The input cloud with rgb values.

Definition at line 175 of file line_rgbd.h.

References cloud_rgb_, and color_gradient_mod_.

## setIntersectionVolumeThreshold()

template<typename PointXYZT, typename PointRGBT = PointXYZT>

void pcl::LineRGBD< PointXYZT, PointRGBT >::setIntersectionVolumeThreshold | ( | const float | threshold = 1.0f | ) |

inline

Sets the threshold for the decision whether two detections of the same template are merged or not.

If ratio between the intersection of the bounding boxes of two detections and the original bounding boxes is larger than the specified threshold then they are merged. If detection A overlaps with detection B and B with C than A, B, and C are merged. Threshold has to be between 0 and 1.

Parameters

  [in] | threshold | The threshold on the ratio between the intersection bounding box and the original bounding box.

Definition at line 154 of file line_rgbd.h.

## Member Data Documentation

## bounding_boxes_

template<typename PointXYZT, typename PointRGBT = PointXYZT>

std::vector<pcl::BoundingBoxXYZ> pcl::LineRGBD< PointXYZT, PointRGBT >::bounding_boxes_

Bounding boxes corresponding to the templates.

Definition at line 301 of file line_rgbd.h.

Referenced by addTemplate(), applyProjectiveDepthICPOnDetections(), computeTransformedTemplatePoints(), createAndAddTemplate(), detect(), detectSemiScaleInvariant(), and loadTemplates().

## cloud_rgb_

template<typename PointXYZT, typename PointRGBT = PointXYZT>

pcl::PointCloud<PointRGBT>::ConstPtr pcl::LineRGBD< PointXYZT, PointRGBT >::cloud_rgb_

RGB point cloud.

Definition at line 296 of file line_rgbd.h.

Referenced by LineRGBD(), and setInputColors().

## cloud_xyz_

template<typename PointXYZT, typename PointRGBT = PointXYZT>

pcl::PointCloud<PointXYZT>::ConstPtr pcl::LineRGBD< PointXYZT, PointRGBT >::cloud_xyz_

XYZ point cloud.

Definition at line 294 of file line_rgbd.h.

Referenced by detect(), detectSemiScaleInvariant(), LineRGBD(), refineDetectionsAlongDepth(), and setInputCloud().

## color_gradient_mod_

template<typename PointXYZT, typename PointRGBT = PointXYZT>

pcl::ColorGradientModality<PointRGBT> pcl::LineRGBD< PointXYZT, PointRGBT >::color_gradient_mod_

Color gradient modality.

Definition at line 289 of file line_rgbd.h.

Referenced by createAndAddTemplate(), detect(), detectSemiScaleInvariant(), LineRGBD(), setGradientMagnitudeThreshold(), and setInputColors().

## detections_

template<typename PointXYZT, typename PointRGBT = PointXYZT>

std::vector<typename pcl::LineRGBD<PointXYZT, PointRGBT>::Detection> pcl::LineRGBD< PointXYZT, PointRGBT >::detections_

Detections from last call of method detect (...).

Definition at line 306 of file line_rgbd.h.

Referenced by alignTemplatePoints(), applyProjectiveDepthICPOnDetections(), computeTransformedTemplatePoints(), detect(), detectSemiScaleInvariant(), findObjectPointIndices(), LineRGBD(), refineDetectionsAlongDepth(), and removeOverlappingDetections().

## linemod_

template<typename PointXYZT, typename PointRGBT = PointXYZT>

pcl::LINEMOD pcl::LineRGBD< PointXYZT, PointRGBT >::linemod_

LINEMOD instance.

Definition at line 287 of file line_rgbd.h.

Referenced by addTemplate(), createAndAddTemplate(), detect(), detectSemiScaleInvariant(), loadTemplates(), and setDetectionThreshold().

## object_ids_

template<typename PointXYZT, typename PointRGBT = PointXYZT>

std::vector<std::size_t> pcl::LineRGBD< PointXYZT, PointRGBT >::object_ids_

Object IDs corresponding to the templates.

Definition at line 303 of file line_rgbd.h.

Referenced by addTemplate(), createAndAddTemplate(), detect(), detectSemiScaleInvariant(), and loadTemplates().

## surface_normal_mod_

template<typename PointXYZT, typename PointRGBT = PointXYZT>

pcl::SurfaceNormalModality<PointXYZT> pcl::LineRGBD< PointXYZT, PointRGBT >::surface_normal_mod_

Surface normal modality.

Definition at line 291 of file line_rgbd.h.

Referenced by createAndAddTemplate(), detect(), detectSemiScaleInvariant(), LineRGBD(), and setInputCloud().

## template_point_clouds_

template<typename PointXYZT, typename PointRGBT = PointXYZT>

pcl::PointCloud<pcl::PointXYZRGBA>::CloudVectorType pcl::LineRGBD< PointXYZT, PointRGBT >::template_point_clouds_

Point clouds corresponding to the templates.

Definition at line 299 of file line_rgbd.h.

Referenced by addTemplate(), applyProjectiveDepthICPOnDetections(), computeTransformedTemplatePoints(), createAndAddTemplate(), and loadTemplates().

The documentation for this class was generated from the following files:

- pcl/recognition/linemod/line_rgbd.h
- pcl/recognition/impl/linemod/line_rgbd.hpp