Hands-on Learning OpenCV with Qt Creator in Ubuntu

"One App to learn them all,

One App to find them,

One App to apply them all and in the Creator bind them."

In this entry the writer presents his results from putting some lessons from the book OpenCV 2 Computer Vision Application Programming Cookbook by Robert Laganière into practice by incorporating the implementations into just one application. The IDE chosen for this is Qt Creator on the OS Ubuntu Gnome (a Linux variant). Initially the application is intended to comply with the principle of K.I.S.S. - Keep It Simple and Stupid. The designated application is supposed to process an input image according to the lessons by respective chapter of the book. The application should consist only of buttons and space for output images. After a certain point, the writer halted and decided to revisit some fundamentals in software architecture.

The cookbook mentioned above is de facto the standard book to learn OpenCV 2 for beginners. Consisting of 10 chapters, the book begins by introducing the steps to install OpenCV, then moves to the creating of OpenCV projects in MS Visual Studio and Qt. Various algorithms for image processing are elaborated in the next chapters - filtering, morphological operations, etc.

Follow @HafizVonPJ

Previously the writer had successfully installed OpenCV in Windows XP and Windows 7. Taking the next level, the writer challenges himself to try OpenCV in a Linux system, which in this case refers to Ubuntu. By using Qt in Ubuntu, the usage of a CMakeLists is replaced with the editing the .pro file for the Qt project. Designing and implementing the GUI file for the application happen within the same platform - Qt Creator.

The result is an application called FingsCV, seen here in the Fig. 0 below.

Fig. 0 - Overview of the application FingsCV (No copyright infringement is intended for the Transformers' Decepticon logo. For academic and non-commercial purpose only).

The Simple and Stupid Design

Fig. 1 below shows the design of the GUI in the designer section of Qt Creator. The GUI consists only of one button to trigger a dialog window to load an input image. The main part of the GUI is only a tab widget that has several tabs that signify the respective chapters.

Fig. 1 - The design of the GUI in Qt Creator.

Results

Fig. 2 below displays the input image used for the operations in the tab Ch. 1 and Ch. 2.

Fig. 2 - Test image (Photo courtesy of a popular cosplayer's Instagram page - no copyright infringement is intended. For academic purpose only).

Chapter 1 of the cookbook is basically about one of the simplest operations in image processing and editing - reflecting or flipping of an image. The result from flipping the test image is shown in Fig. 3 below.

Fig. 3 - The output of flipping the test image from Fig. 2.

Then in Chapter 2, more operations are introduced  - putting salt and pepper (white and black pixels) effect, color reducing, and image sharpening. Fig. 4 below shows the after-effect of random and synthetic white pixels (the 'salt') on the test image.

Fig. 4 - Test image with added random salt - marked by the tiny white pixels.

Discussion

In the beginning, the writer used OpenCV's HighGUI for visualization of images. This had a conflict with Qt, while the latter has its own built-in implementations for image displays. Albeit no error and just warnings, the writer switched to just use Qt's features such as QImage, QPixmap, etc for the visualizations.

The writer decided to stop this development until Chapter 2, because the Chapter 3 requires more interactions via the GUI - input parameters, signal buttons, etc. This ought to be done in a separate application. To implement the next lessons, the application should be more interactive and pragmatic. 

Future Prospects

The predicaments from Chapter 3 led the writer to refer a book from his college's library - Software Architecture - Foundations, Theory, and Practice by Richard N. Taylor.

From this book by R. Taylor, it looks like the process of software development shouldn't be taken lightly. In simple words, the development of a software can be summed up in these steps:

• Listing of the requirements
• Designing
• Implementing - the coding
• Analysis and testing
• Evolution and maintenance

Thus, the K.I.S.S. should be modified to 'Keep It Stupid and Simple - enough'. The last word 'enough' implies that a certain degree of requirements must be firstly met.

At the moment, the writer considers that every version of this app should be dedicated to only one chapter of the cookbook. Therefore separate Qt projects for all the chapters of the cookbook.

FingKinector 2.0 - on Qt Creator with PCL and OpenCV

How can one build a Qt project that implements both OpenCV and Point Cloud Library (PCL)? The writer aims to have the best of both worlds available on the same platform. This article briefs on the writer's attempt to achieve that objective with one CMakeLists file and the development within the Qt Creator. The project in this article is FingKinector 2.0, the writer's updated version of a simple application for Microsoft Kinect sensor.

Follow @HafizVonPJ

Mainly the writer deals with 3-dimensional point clouds, which is why the PCL is used for most of the writer's C++ code. But when it comes to situations that require linear algebra calculations especially matrix operations (e.g. transformation), the writer finds OpenCV library to be convenient. The OpenCV library enables one to deal with n-dimensional images, which one usually treats them as matrices.

For personal/independent projects, the writer aims to develop one simple application on the platform of Qt Creator and uses both PCL and OpenCV. Previously the writer has developed the 'stupid and simple' app: FingKinector. This first version only had access to the joint motor functions of the Kinect sensor. Furthermore it was developed on Microsoft Visual Studio. For the sake of variety, the writer thought this time the upgrading of this app should be done on Qt Creator.

CMake Build Process

The following images show the step by step in the build process of the software for FingKinector 2.0, within the Qt Creator, instead of the CMake GUI.

Step 1: Open file CMakeLists.txt (Fig. 1)

Figure 1: Open File - CMakeLists.txt

Step 2: Specify the directory for the build (Fig. 2)

Figure 2: Specify the directory for the build process

Step 3: Run CMake (Figs. 3.1 and 3.2)

Figure 3.1: Run CMake

Figure 3.2: Finish the build process

Step 4: Done - start coding! (Fig. 4)

Figure 4: Start coding on Qt Creator

Source Codes

The writer finds it prudent to divide the codes for Kinect motor functions and its image processing in different 2 separate files. The figure (Fig. 5) shows the source files for this application:

Figure 5: Source codes and other files for the project

1) main: contains the function int main()

2) FingKinectorWindow: manages the actions/signals on the GUI

3) KinectFunkts: manages Kinect initialization and frame grabber

4) PCLOpenCV: manages the processing of 3D-point clouds and its visualization.

The writer has decided not to make these source codes available for others. However the writer welcomes suggestions and questions from all readers.

New Features

In the first version of FingKinector, one could only toy with the joint motor feature of the Kinect sensor. For this new version, FingKinector offers two options for point cloud acquisition:

i) the fundamental single mode / one shot, and

ii) convoluted triple shots - at 3 different angles: 27°, 0° and -27°.

The following figures 6.1 until 6.3 show the feature of every tab on the app FingKinector.

Fig. 6.1: FingKinector - Initialization

Figure 6.2: FingKinector - Motor functions

Figure 6.3: FingKinector - Visualization

The latter mode is actually redundant, but the writer was curious about the outcome of such acquisition.

Future prospects

The writer would add two more features for this app in the future - live view of 3D-point cloud and display of range image.

Demonstrations

MATLAB Code to Generate a Chessboard

Follow @HafizVonPJ

In the last 2 years, the writer attended a lecture on Linear Filters in Image Processing. Among the various assignments was to write a function which generates a chessboard/checkerboard, which then can be visualized as a conventional 2D image. This is a tutorial on how to apply a function called 'schachbrett' (the German word for chessboard) in MATLAB [1].

Why chessboard?

Figure 1: An artificially generated image of chessboard (256 x 256 pixel)

The chessboard (Figure 1) was deemed an interesting object to investigate frequencies after applying Fourier Transform. The test would looked like Figure 2 and Figure 3 below.

Figure 2: The real part of the Fourier Transform of lohSchahmat

Figure 3: The real part of the Fourier Transform of lohSchahmat (as a 3D-mesh plot)

The results would look finer if the writer had applied:

1. a higher number of pixel and/or
2. a higher number of fields. 

The function schachbrett()

This function takes 2 arguments - namely the no. of pixel and and the no. of fields to be displayed.

function [chessBoard] = schachbrett(noPxl,noField)

And the output is - chessBoard.

For security, the writer set default values as follows:

% Setting the default no. of pixels

if noPxl <= 0

    noPxl = 10;

end

% Dimension of the board => noField x noField

if noField <= 0

    noField = 2;

end

The here is where the magic happens:

boardTemp = 255*[ones(noPxl) zeros(noPxl); zeros(noPxl) ones(noPxl)];

chessBoard = repmat(boardTemp,noField/2,noField/2);

Firstly a 2x2 board of ones and zeros are generated, which resembles an identity matrix, except that this matrix is NOT an identity matrix. Then it is multiplied by the value of 255, so that all values of 1 become 255 (the color white in 8-bit grayscale).

Then the matrix is replicated with the help of a built-in MATLAB function called repmat() - replicated as half as the number of fields given as the argument noField in schachbrett() above.

How to apply the function schachbrett()

This step is very easy just like below, which has generated the image in Figure 1 above:

noPixel = 2^5;

noLineColumn = 2^3;

lohSchahmat = schachbrett(noPixel, noLineColumn);

figure('Name', 'BetHaMikdash'),

imshow(lohSchahmat)

title('Rosfah Schlomo')

That is all. Have fun with coding in MATLAB!

Remarks:

[1]. MATLAB is a registered trademark of The Mathworks, Inc. The code above was strictly used for academical and research purpose.