3D Point Cloud Processing with PCL

"What's better than 2D?"

Three dimensional (3D) point cloud can be understood as a set of points in the Euclidean coordinate system, with every point has X-, Y-, and Z-coordinates. Combined with the corresponding normal vectors or normals, this set forms oriented points - providing 6 degrees of freedom (DOF). This article aims to briefly introduce readers to the 3D point cloud processing with an open source library called Point Cloud Library (PCL). Two examples of using PCL are given and visualized.

Follow @HafizVonPJ

A 3D point cloud can be acquired using range sensors such as Microsoft Kinect, ASUS Xtion, or laser scanners. These devices apply either triangulation or time-of-flight (TOF). Apart from that, using the autofocus system on conventional industrial cameras can also deliver 3D point clouds.

To process and visualize 3D point clouds, PCL offers various modules, e.g., filtering, segmentation, I/O, data structures, and visualization. The author has been tasked to implement several of these features in the open source application for image processing called MILAN. The implementations are available within one particular plug-in of MILAN. The following images are examples of using MILAN with PCL.

Figure 1 - The features of PCL in MILAN are accessible via a toolbox docked to the right side as seen above. This is also where the Kinect sensor can be used and controlled.

Figure 2: PCL Downsampling Filter, to reduce the size of an input cloud (left - 63792 vertices) to an output (right - 20462 vertices).

Figure 3: PCL Euclidean Cluster Segmentation, to segmentize spatially isolated point clouds.

Before using MILAN, the author has used the PCL tutorials and implemented them in separate Microsoft Visual Studio (MSVS) 2008 projects. The Service Packet 1 (SP 1) of this IDE needs to be installed, otherwise the developers can only code in Release mode.

Another alternative to MSVS is the Qt Creator. The author has covered this in one post on the app FingKinector 2.0. It was developed with Qt 4 along OpenCV 2.4.5. Without writing any .PRO file for the Qt project, the author used a CMakeLists.txt entirely to build this Qt project.

Hands-on Learning OpenCV with Qt Creator in Ubuntu

"One App to learn them all,

One App to find them,

One App to apply them all and in the Creator bind them."

In this entry the writer presents his results from putting some lessons from the book OpenCV 2 Computer Vision Application Programming Cookbook by Robert Laganière into practice by incorporating the implementations into just one application. The IDE chosen for this is Qt Creator on the OS Ubuntu Gnome (a Linux variant). Initially the application is intended to comply with the principle of K.I.S.S. - Keep It Simple and Stupid. The designated application is supposed to process an input image according to the lessons by respective chapter of the book. The application should consist only of buttons and space for output images. After a certain point, the writer halted and decided to revisit some fundamentals in software architecture.

The cookbook mentioned above is de facto the standard book to learn OpenCV 2 for beginners. Consisting of 10 chapters, the book begins by introducing the steps to install OpenCV, then moves to the creating of OpenCV projects in MS Visual Studio and Qt. Various algorithms for image processing are elaborated in the next chapters - filtering, morphological operations, etc.

Follow @HafizVonPJ

Previously the writer had successfully installed OpenCV in Windows XP and Windows 7. Taking the next level, the writer challenges himself to try OpenCV in a Linux system, which in this case refers to Ubuntu. By using Qt in Ubuntu, the usage of a CMakeLists is replaced with the editing the .pro file for the Qt project. Designing and implementing the GUI file for the application happen within the same platform - Qt Creator.

The result is an application called FingsCV, seen here in the Fig. 0 below.

Fig. 0 - Overview of the application FingsCV (No copyright infringement is intended for the Transformers' Decepticon logo. For academic and non-commercial purpose only).

The Simple and Stupid Design

Fig. 1 below shows the design of the GUI in the designer section of Qt Creator. The GUI consists only of one button to trigger a dialog window to load an input image. The main part of the GUI is only a tab widget that has several tabs that signify the respective chapters.

Fig. 1 - The design of the GUI in Qt Creator.

Results

Fig. 2 below displays the input image used for the operations in the tab Ch. 1 and Ch. 2.

Fig. 2 - Test image (Photo courtesy of a popular cosplayer's Instagram page - no copyright infringement is intended. For academic purpose only).

Chapter 1 of the cookbook is basically about one of the simplest operations in image processing and editing - reflecting or flipping of an image. The result from flipping the test image is shown in Fig. 3 below.

Fig. 3 - The output of flipping the test image from Fig. 2.

Then in Chapter 2, more operations are introduced  - putting salt and pepper (white and black pixels) effect, color reducing, and image sharpening. Fig. 4 below shows the after-effect of random and synthetic white pixels (the 'salt') on the test image.

Fig. 4 - Test image with added random salt - marked by the tiny white pixels.

Discussion

In the beginning, the writer used OpenCV's HighGUI for visualization of images. This had a conflict with Qt, while the latter has its own built-in implementations for image displays. Albeit no error and just warnings, the writer switched to just use Qt's features such as QImage, QPixmap, etc for the visualizations.

The writer decided to stop this development until Chapter 2, because the Chapter 3 requires more interactions via the GUI - input parameters, signal buttons, etc. This ought to be done in a separate application. To implement the next lessons, the application should be more interactive and pragmatic. 

Future Prospects

The predicaments from Chapter 3 led the writer to refer a book from his college's library - Software Architecture - Foundations, Theory, and Practice by Richard N. Taylor.

From this book by R. Taylor, it looks like the process of software development shouldn't be taken lightly. In simple words, the development of a software can be summed up in these steps:

• Listing of the requirements
• Designing
• Implementing - the coding
• Analysis and testing
• Evolution and maintenance

Thus, the K.I.S.S. should be modified to 'Keep It Stupid and Simple - enough'. The last word 'enough' implies that a certain degree of requirements must be firstly met.

At the moment, the writer considers that every version of this app should be dedicated to only one chapter of the cookbook. Therefore separate Qt projects for all the chapters of the cookbook.

MATLAB Code to Generate a Chessboard

Follow @HafizVonPJ

In the last 2 years, the writer attended a lecture on Linear Filters in Image Processing. Among the various assignments was to write a function which generates a chessboard/checkerboard, which then can be visualized as a conventional 2D image. This is a tutorial on how to apply a function called 'schachbrett' (the German word for chessboard) in MATLAB [1].

Why chessboard?

Figure 1: An artificially generated image of chessboard (256 x 256 pixel)

The chessboard (Figure 1) was deemed an interesting object to investigate frequencies after applying Fourier Transform. The test would looked like Figure 2 and Figure 3 below.

Figure 2: The real part of the Fourier Transform of lohSchahmat

Figure 3: The real part of the Fourier Transform of lohSchahmat (as a 3D-mesh plot)

The results would look finer if the writer had applied:

1. a higher number of pixel and/or
2. a higher number of fields. 

The function schachbrett()

This function takes 2 arguments - namely the no. of pixel and and the no. of fields to be displayed.

function [chessBoard] = schachbrett(noPxl,noField)

And the output is - chessBoard.

For security, the writer set default values as follows:

% Setting the default no. of pixels

if noPxl <= 0

    noPxl = 10;

end

% Dimension of the board => noField x noField

if noField <= 0

    noField = 2;

end

The here is where the magic happens:

boardTemp = 255*[ones(noPxl) zeros(noPxl); zeros(noPxl) ones(noPxl)];

chessBoard = repmat(boardTemp,noField/2,noField/2);

Firstly a 2x2 board of ones and zeros are generated, which resembles an identity matrix, except that this matrix is NOT an identity matrix. Then it is multiplied by the value of 255, so that all values of 1 become 255 (the color white in 8-bit grayscale).

Then the matrix is replicated with the help of a built-in MATLAB function called repmat() - replicated as half as the number of fields given as the argument noField in schachbrett() above.

How to apply the function schachbrett()

This step is very easy just like below, which has generated the image in Figure 1 above:

noPixel = 2^5;

noLineColumn = 2^3;

lohSchahmat = schachbrett(noPixel, noLineColumn);

figure('Name', 'BetHaMikdash'),

imshow(lohSchahmat)

title('Rosfah Schlomo')

That is all. Have fun with coding in MATLAB!

Remarks:

[1]. MATLAB is a registered trademark of The Mathworks, Inc. The code above was strictly used for academical and research purpose.