Structure-based object representation and classification in mobile robotics through a Microsoft Kinect

•A new approach for object representation and classification is proposed.•We rely on considerations about the structure of furniture-sized objects.•The 3D point cloud returned by the Kinect is segmented into a set of clusters.•Objects are represented by expressing mutual relationships between clusters.•The approach is validated through experiments with real data.

A new approach enabling a mobile robot to recognize and classify furniture-like objects composed of assembled parts using a Microsoft Kinect is presented. Starting from considerations about the structure of furniture-like objects, i.e., objects which can play a role in the course of a mobile robot mission, the 3D point cloud returned by the Kinect is first segmented into a set of “almost convex” clusters. Objects are then represented by means of a graph expressing mutual relationships between such clusters. Off-line, snapshots of the same object taken from different positions are processed and merged, in order to produce multiple-view models that are used to populate a database. On-line, as soon as a new object is observed, a run-time window of subsequent snapshots is used to search for a correspondence in the database.Experiments validating the approach with a set of objects (i.e., chairs, tables, but also other robots) are reported and discussed in detail.