Hierarchical Semantic Labeling in Point Cloud Scenes Using a Probabilistic Grammar

Abstract

The amount of 3D data available online has been growing recently. While 3D scenes are commonly represented by hierarchies of mesh objects, the point cloud format is becoming more popular due to the proliferation of cheap and hack-able consumer depth sensors such as Microsoft's Kinect. These scenes are relatively easy to construct with such sensors, and could be used in many interesting data- driven applications. Unfortunately, most point cloud scenes are unsegmented and unlabeled, existing as one large collection of unstructured points. Thus, it would be useful to be able to automatically identify and semantically label objects in such scenes. Previous work has been done to accomplish this goal in mesh scenes, but this work has not yet been expanded to include point clouds. In this paper we adapt a technique that uses labeled training scenes to generate a hierarchical probabilistic scene grammar used to parse new unstructured scenes. We have built a data pipeline for the analysis of point cloud data sets using this technique and provided preliminary results with regards to performance and accuracy. The results for a limited number of test scenes are promising and indicate that this method is likely very effective at labeling this sort of data.