3D Point Cloud Processing with PCL

"What's better than 2D?"

Three dimensional (3D) point cloud can be understood as a set of points in the Euclidean coordinate system, with every point has X-, Y-, and Z-coordinates. Combined with the corresponding normal vectors or normals, this set forms oriented points - providing 6 degrees of freedom (DOF). This article aims to briefly introduce readers to the 3D point cloud processing with an open source library called Point Cloud Library (PCL). Two examples of using PCL are given and visualized.

Follow @HafizVonPJ

A 3D point cloud can be acquired using range sensors such as Microsoft Kinect, ASUS Xtion, or laser scanners. These devices apply either triangulation or time-of-flight (TOF). Apart from that, using the autofocus system on conventional industrial cameras can also deliver 3D point clouds.

To process and visualize 3D point clouds, PCL offers various modules, e.g., filtering, segmentation, I/O, data structures, and visualization. The author has been tasked to implement several of these features in the open source application for image processing called MILAN. The implementations are available within one particular plug-in of MILAN. The following images are examples of using MILAN with PCL.

Figure 1 - The features of PCL in MILAN are accessible via a toolbox docked to the right side as seen above. This is also where the Kinect sensor can be used and controlled.

Figure 2: PCL Downsampling Filter, to reduce the size of an input cloud (left - 63792 vertices) to an output (right - 20462 vertices).

Figure 3: PCL Euclidean Cluster Segmentation, to segmentize spatially isolated point clouds.

Before using MILAN, the author has used the PCL tutorials and implemented them in separate Microsoft Visual Studio (MSVS) 2008 projects. The Service Packet 1 (SP 1) of this IDE needs to be installed, otherwise the developers can only code in Release mode.

Another alternative to MSVS is the Qt Creator. The author has covered this in one post on the app FingKinector 2.0. It was developed with Qt 4 along OpenCV 2.4.5. Without writing any .PRO file for the Qt project, the author used a CMakeLists.txt entirely to build this Qt project.

FingKinector 2.0 - on Qt Creator with PCL and OpenCV

How can one build a Qt project that implements both OpenCV and Point Cloud Library (PCL)? The writer aims to have the best of both worlds available on the same platform. This article briefs on the writer's attempt to achieve that objective with one CMakeLists file and the development within the Qt Creator. The project in this article is FingKinector 2.0, the writer's updated version of a simple application for Microsoft Kinect sensor.

Follow @HafizVonPJ

Mainly the writer deals with 3-dimensional point clouds, which is why the PCL is used for most of the writer's C++ code. But when it comes to situations that require linear algebra calculations especially matrix operations (e.g. transformation), the writer finds OpenCV library to be convenient. The OpenCV library enables one to deal with n-dimensional images, which one usually treats them as matrices.

For personal/independent projects, the writer aims to develop one simple application on the platform of Qt Creator and uses both PCL and OpenCV. Previously the writer has developed the 'stupid and simple' app: FingKinector. This first version only had access to the joint motor functions of the Kinect sensor. Furthermore it was developed on Microsoft Visual Studio. For the sake of variety, the writer thought this time the upgrading of this app should be done on Qt Creator.

CMake Build Process

The following images show the step by step in the build process of the software for FingKinector 2.0, within the Qt Creator, instead of the CMake GUI.

Step 1: Open file CMakeLists.txt (Fig. 1)

Figure 1: Open File - CMakeLists.txt

Step 2: Specify the directory for the build (Fig. 2)

Figure 2: Specify the directory for the build process

Step 3: Run CMake (Figs. 3.1 and 3.2)

Figure 3.1: Run CMake

Figure 3.2: Finish the build process

Step 4: Done - start coding! (Fig. 4)

Figure 4: Start coding on Qt Creator

Source Codes

The writer finds it prudent to divide the codes for Kinect motor functions and its image processing in different 2 separate files. The figure (Fig. 5) shows the source files for this application:

Figure 5: Source codes and other files for the project

1) main: contains the function int main()

2) FingKinectorWindow: manages the actions/signals on the GUI

3) KinectFunkts: manages Kinect initialization and frame grabber

4) PCLOpenCV: manages the processing of 3D-point clouds and its visualization.

The writer has decided not to make these source codes available for others. However the writer welcomes suggestions and questions from all readers.

New Features

In the first version of FingKinector, one could only toy with the joint motor feature of the Kinect sensor. For this new version, FingKinector offers two options for point cloud acquisition:

i) the fundamental single mode / one shot, and

ii) convoluted triple shots - at 3 different angles: 27°, 0° and -27°.

The following figures 6.1 until 6.3 show the feature of every tab on the app FingKinector.

Fig. 6.1: FingKinector - Initialization

Figure 6.2: FingKinector - Motor functions

Figure 6.3: FingKinector - Visualization

The latter mode is actually redundant, but the writer was curious about the outcome of such acquisition.

Future prospects

The writer would add two more features for this app in the future - live view of 3D-point cloud and display of range image.

Demonstrations