Hands-on Learning OpenCV with Qt Creator in Ubuntu

"One App to learn them all,

One App to find them,

One App to apply them all and in the Creator bind them."

In this entry the writer presents his results from putting some lessons from the book OpenCV 2 Computer Vision Application Programming Cookbook by Robert Laganière into practice by incorporating the implementations into just one application. The IDE chosen for this is Qt Creator on the OS Ubuntu Gnome (a Linux variant). Initially the application is intended to comply with the principle of K.I.S.S. - Keep It Simple and Stupid. The designated application is supposed to process an input image according to the lessons by respective chapter of the book. The application should consist only of buttons and space for output images. After a certain point, the writer halted and decided to revisit some fundamentals in software architecture.

The cookbook mentioned above is de facto the standard book to learn OpenCV 2 for beginners. Consisting of 10 chapters, the book begins by introducing the steps to install OpenCV, then moves to the creating of OpenCV projects in MS Visual Studio and Qt. Various algorithms for image processing are elaborated in the next chapters - filtering, morphological operations, etc.

Follow @HafizVonPJ

Previously the writer had successfully installed OpenCV in Windows XP and Windows 7. Taking the next level, the writer challenges himself to try OpenCV in a Linux system, which in this case refers to Ubuntu. By using Qt in Ubuntu, the usage of a CMakeLists is replaced with the editing the .pro file for the Qt project. Designing and implementing the GUI file for the application happen within the same platform - Qt Creator.

The result is an application called FingsCV, seen here in the Fig. 0 below.

Fig. 0 - Overview of the application FingsCV (No copyright infringement is intended for the Transformers' Decepticon logo. For academic and non-commercial purpose only).

The Simple and Stupid Design

Fig. 1 below shows the design of the GUI in the designer section of Qt Creator. The GUI consists only of one button to trigger a dialog window to load an input image. The main part of the GUI is only a tab widget that has several tabs that signify the respective chapters.

Fig. 1 - The design of the GUI in Qt Creator.

Results

Fig. 2 below displays the input image used for the operations in the tab Ch. 1 and Ch. 2.

Fig. 2 - Test image (Photo courtesy of a popular cosplayer's Instagram page - no copyright infringement is intended. For academic purpose only).

Chapter 1 of the cookbook is basically about one of the simplest operations in image processing and editing - reflecting or flipping of an image. The result from flipping the test image is shown in Fig. 3 below.

Fig. 3 - The output of flipping the test image from Fig. 2.

Then in Chapter 2, more operations are introduced  - putting salt and pepper (white and black pixels) effect, color reducing, and image sharpening. Fig. 4 below shows the after-effect of random and synthetic white pixels (the 'salt') on the test image.

Fig. 4 - Test image with added random salt - marked by the tiny white pixels.

Discussion

In the beginning, the writer used OpenCV's HighGUI for visualization of images. This had a conflict with Qt, while the latter has its own built-in implementations for image displays. Albeit no error and just warnings, the writer switched to just use Qt's features such as QImage, QPixmap, etc for the visualizations.

The writer decided to stop this development until Chapter 2, because the Chapter 3 requires more interactions via the GUI - input parameters, signal buttons, etc. This ought to be done in a separate application. To implement the next lessons, the application should be more interactive and pragmatic. 

Future Prospects

The predicaments from Chapter 3 led the writer to refer a book from his college's library - Software Architecture - Foundations, Theory, and Practice by Richard N. Taylor.

From this book by R. Taylor, it looks like the process of software development shouldn't be taken lightly. In simple words, the development of a software can be summed up in these steps:

• Listing of the requirements
• Designing
• Implementing - the coding
• Analysis and testing
• Evolution and maintenance

Thus, the K.I.S.S. should be modified to 'Keep It Stupid and Simple - enough'. The last word 'enough' implies that a certain degree of requirements must be firstly met.

At the moment, the writer considers that every version of this app should be dedicated to only one chapter of the cookbook. Therefore separate Qt projects for all the chapters of the cookbook.

FingKinector 2.0 - on Qt Creator with PCL and OpenCV

How can one build a Qt project that implements both OpenCV and Point Cloud Library (PCL)? The writer aims to have the best of both worlds available on the same platform. This article briefs on the writer's attempt to achieve that objective with one CMakeLists file and the development within the Qt Creator. The project in this article is FingKinector 2.0, the writer's updated version of a simple application for Microsoft Kinect sensor.

Follow @HafizVonPJ

Mainly the writer deals with 3-dimensional point clouds, which is why the PCL is used for most of the writer's C++ code. But when it comes to situations that require linear algebra calculations especially matrix operations (e.g. transformation), the writer finds OpenCV library to be convenient. The OpenCV library enables one to deal with n-dimensional images, which one usually treats them as matrices.

For personal/independent projects, the writer aims to develop one simple application on the platform of Qt Creator and uses both PCL and OpenCV. Previously the writer has developed the 'stupid and simple' app: FingKinector. This first version only had access to the joint motor functions of the Kinect sensor. Furthermore it was developed on Microsoft Visual Studio. For the sake of variety, the writer thought this time the upgrading of this app should be done on Qt Creator.

CMake Build Process

The following images show the step by step in the build process of the software for FingKinector 2.0, within the Qt Creator, instead of the CMake GUI.

Step 1: Open file CMakeLists.txt (Fig. 1)

Figure 1: Open File - CMakeLists.txt

Step 2: Specify the directory for the build (Fig. 2)

Figure 2: Specify the directory for the build process

Step 3: Run CMake (Figs. 3.1 and 3.2)

Figure 3.1: Run CMake

Figure 3.2: Finish the build process

Step 4: Done - start coding! (Fig. 4)

Figure 4: Start coding on Qt Creator

Source Codes

The writer finds it prudent to divide the codes for Kinect motor functions and its image processing in different 2 separate files. The figure (Fig. 5) shows the source files for this application:

Figure 5: Source codes and other files for the project

1) main: contains the function int main()

2) FingKinectorWindow: manages the actions/signals on the GUI

3) KinectFunkts: manages Kinect initialization and frame grabber

4) PCLOpenCV: manages the processing of 3D-point clouds and its visualization.

The writer has decided not to make these source codes available for others. However the writer welcomes suggestions and questions from all readers.

New Features

In the first version of FingKinector, one could only toy with the joint motor feature of the Kinect sensor. For this new version, FingKinector offers two options for point cloud acquisition:

i) the fundamental single mode / one shot, and

ii) convoluted triple shots - at 3 different angles: 27°, 0° and -27°.

The following figures 6.1 until 6.3 show the feature of every tab on the app FingKinector.

Fig. 6.1: FingKinector - Initialization

Figure 6.2: FingKinector - Motor functions

Figure 6.3: FingKinector - Visualization

The latter mode is actually redundant, but the writer was curious about the outcome of such acquisition.

Future prospects

The writer would add two more features for this app in the future - live view of 3D-point cloud and display of range image.

Demonstrations

Welcome to Germany...and Welcome to the Suck

First of all - the second phrase in the title above is just a pun by the writer. The writer dedicates this blog post to the group of 41 products of German A-Level Preparatory Program (GAPP), which arrived to Germany via its Frankfurt International Airport a.k.a. Fraport on the last March 15, 2014, and to the current GAPP students at the German-Malaysian Institute (GMI) in Bangi, Malaysia. This article is intended to provide simple yet sufficient information for them about the main challenge that awaits them in Germany once they arrive here: the TestDaF.

Follow @HafizVonPJ

On Saturday March 15, 2014 - a group of 41 future students arrived at the Fraport. Once they came out from the arrival gate in Fraport Terminal 2, they were led to an area in the Terminal 2. There they were given short briefings by 2 officers from the General Consulate of Malaysia in Frankfurt am Main. Afterwards they were separated to 4 groups - each group would then journey to a different state in Germany: Bayern (Bavaria), Sachsen (Saxony), Nordrhein-Westfalen (North-Rhein Westphalia / NRW) and Hessen (Hesse).  At each place they must then undergo a preparation course for the ultimate German language test for them - the TestDaF / Test Deutsch als Fremdsprache (German as Foreign Language).

The writer congratulates them for making it this far and hope they have adjusted themselves well in their first week in Germany.

Financial Management during the Language Course

In a briefing by one of the officers, students were advised to manage their allowance smartly and told to spare a certain amount of money from their early allowance for certain reasons. From their monthly allowance they must pay for living costs: mostly for room rent, health insurance, cellphone bills and of course foods and drinks. Here the writer would like to urge these students to refrain from spending on large (read: expensive) items such as TV-Monitor, desktop PC, musical instruments etc, which would then trouble them during the process of relocation to their respective college.

The process of moving into a new rent room always involves a big sum of money. They need to consider the costs of transportation, deposit of the rent (which is usually 2 or 3 month of monthly rent), the first month rent, new furniture and so on, hence the reminder of sparing some money from their early allowance. Apart from those, they must also be prepared to pay the semester fee (€ 200 and above).

The TestDaF

From here onward, the writer addresses the 41 students as TestDaF candidates. The writer welcomes comments from other fellow students who have also experienced the TestDaF, especially those who scored 16 points and above, to share their opinions and tips.

The writer took a liberty to briefly view the website of TestDaF-Institut [1], to refresh himself on information regarding the TestDaF. The following is the summary on TestDaF along with tips/suggestions by the writer. TestDaF candidates are hereby recommended to read further information and tips at the official website (the website is unforgiving - everything is in German, but that is the whole point - to test your German language competence). The writer predicts that all 41 candidates will do their TestDaF on July 15, 2014 [2].

1) The Structure of TestDaF

The following image (Fig. 1) shows that TestDaF is divided to 4 parts in chronological order:

• Reading comprehension (Leseverstehen / LV),
• Listening comprehension (Hörverstehen / HV),
• Text composition (Schriftlicher Ausdruck / SA) and
• Oral (Mündlicher Ausdruck / MA).

Figure 1: The structure of TestDaF [3]

The total time of the test is 3 hours and 10 minutes, without the break time. The answer papers with the recording of the oral test (yes, it will be recorded) would then be sent to the TestDaF-Institut in Bochum for evaluation [3]. It takes at least 6 weeks until the result will be released.

2) Grading

The test result on TestDaF-certificate will be ordered in 3 grade levels to all 4 parts:

• TestDaF-Niveaustufe 3 (TDN 3)
• TestDaF-Niveaustufe 4 (TDN 4)
• TestDaF-Niveaustufe 5 (TDN 5)

The highest grade is TDN 5 - which makes the total highest grade overall is 20. If a grade is below 3, then it will be displayed as 'unter TDN 3' - which is obviously undesired by any German university. In comparison to certain English language certificates which are only valid for 2 years, a TestDaF certificate is described as 'valid without limit' [3].

3) The Required Grade for Enrollment

A candidate is considered to have passed the TestDaF if the total grade or points is 12 (TDN 3 multiplied by 4). To the writer's knowledge, so far only colleges in Bayern that accept such result. The writer would like to shift the reader's attention to colleges in other states, particularly in Hessen. This can be regarded as a highly significant case study.

In the process of application for a placement at a Hessian college, generally it is sufficient for an applicant to provide that she/he has acquired 15 points.

HOWEVER, upon enrollment the candidate must provide a TestDaF-certificate with minimum 16 points, with minimum TDN 4 in every part of the test (4 times TDN 4). The reason behind this is: applicants with 15 points are expected to re-sit the TestDaF and improve their results after sending their college application.

To clarify this matter, last year the writer made a phone call to the Director of International Affairs of Technische Hochschule Mittelhessen and consulted the advisor-in-charge for foreign students of Hochschule Darmstadt. A common point from them was the logic of requiring every applicant to get 16 points and above. They insisted that foreign students should be competent in their German language, to provide the very basic strength in doing an undergraduate degree in Germany. The writer could only agree with them.

4) Tips/Suggestions

As a starter, the writer suggests that TestDaF candidates go through the TestDaF website and download everything available to help them: from the sample test papers in PDF format to the audio files for HV part in MP3 format.

Be active when possessing all those materials - analyze them.

Read articles that come along with tables and graphs. Many phrases to describe the statistics are so useful that they can be applied later especially in the part SA of the TestDaF.

Candidates should also always practice to answer the oral questions effectively. Their all hi-tech gadgets should be utilized to record their voices and replay them. Do mock oral tests during free time.

Stop listening to MP3s when candidates are in places full of German-speaking people. Pay attention to their pronunciation, accents and tones. Try to emulate their accent. Try to slightly understand the topic of their conversation - it may be a dry humor, or may be about their music taste, outing plan, their train destination, how old they are, what kind of pets they have, the current political situation in EU or elsewhere, etc. Whatever. Anything.

It is important to remember why all of this matters so much. The candidates should bear in mind that they will have to attend countless lectures, spend hours to write lab reports, digest lecture scripts and notes - everything in German. The most challenging part would then be to write the final year thesis (Abschlussarbeit), for which bachelor candidates are required to write in impeccable German, before presenting it during a colloquium for final evaluation.

Conclusion

The candidates should fully utilize their 4 month-course to prepare themselves for the TestDaF. The writer urge them to NOT regard this 4-month-period as a rewarding time for them after passing the GAPP course in Malaysia. Instead they should take this as an opportunity to put their German knowledge in practice. No more simulation of dialog - but real-time conversation with native speakers. The candidates are also urged to aim for 16 points or above, for it shall open many doors to them afterward. The writer wishes all the best to the TestDaF candidates and again: 'Welcome To The Suck'*.

Sources:

[1] TestDaF-Institut official website

[2] Available dates for TestDaF

[3] Overview of the TestDaF

*borrowed from the catchphrase of the film Jarhead.

MATLAB Code to Generate a Chessboard

Follow @HafizVonPJ

In the last 2 years, the writer attended a lecture on Linear Filters in Image Processing. Among the various assignments was to write a function which generates a chessboard/checkerboard, which then can be visualized as a conventional 2D image. This is a tutorial on how to apply a function called 'schachbrett' (the German word for chessboard) in MATLAB [1].

Why chessboard?

Figure 1: An artificially generated image of chessboard (256 x 256 pixel)

The chessboard (Figure 1) was deemed an interesting object to investigate frequencies after applying Fourier Transform. The test would looked like Figure 2 and Figure 3 below.

Figure 2: The real part of the Fourier Transform of lohSchahmat

Figure 3: The real part of the Fourier Transform of lohSchahmat (as a 3D-mesh plot)

The results would look finer if the writer had applied:

1. a higher number of pixel and/or
2. a higher number of fields. 

The function schachbrett()

This function takes 2 arguments - namely the no. of pixel and and the no. of fields to be displayed.

function [chessBoard] = schachbrett(noPxl,noField)

And the output is - chessBoard.

For security, the writer set default values as follows:

% Setting the default no. of pixels

if noPxl <= 0

    noPxl = 10;

end

% Dimension of the board => noField x noField

if noField <= 0

    noField = 2;

end

The here is where the magic happens:

boardTemp = 255*[ones(noPxl) zeros(noPxl); zeros(noPxl) ones(noPxl)];

chessBoard = repmat(boardTemp,noField/2,noField/2);

Firstly a 2x2 board of ones and zeros are generated, which resembles an identity matrix, except that this matrix is NOT an identity matrix. Then it is multiplied by the value of 255, so that all values of 1 become 255 (the color white in 8-bit grayscale).

Then the matrix is replicated with the help of a built-in MATLAB function called repmat() - replicated as half as the number of fields given as the argument noField in schachbrett() above.

How to apply the function schachbrett()

This step is very easy just like below, which has generated the image in Figure 1 above:

noPixel = 2^5;

noLineColumn = 2^3;

lohSchahmat = schachbrett(noPixel, noLineColumn);

figure('Name', 'BetHaMikdash'),

imshow(lohSchahmat)

title('Rosfah Schlomo')

That is all. Have fun with coding in MATLAB!

Remarks:

[1]. MATLAB is a registered trademark of The Mathworks, Inc. The code above was strictly used for academical and research purpose.

FingKinector - Interacting with the Kinect Motor

Follow @HafizVonPJ

The pivot which connects the Kinect sensor to its base is motorized. This post is about an app called 'FingKinector', which is developed to utilize that motorized pivot without having to rotate the Kinect sensor manually. This is tested only on the Kinect sensor for XBOX 360 and developed for Windows 7.

"K.I.S.S. = Keep It Stupid, Simple"

Image 1: Kinect Sensor for XBOX 360, equipped with a wide-angle lens.

To be able to control the Kinect sensor from a PC, you will need an AC-adapter for your Kinect. It costs around € 8.00 on the Amazon. It has 2 outputs: the orange plug connector is used to supply the Kinect with 12 V at 1 A, whereas the second output is a regular USB connector [1].

If your Kinect is connected to your PC for the first time, by default Windows will attempt to automatically download and install the driver for that Kinect, which comes from PrimeSense. PrimeSense is the developer of the range camera in the Kinect sensor and it is an Israeli 3D sensing technology. As of this time - PrimeSense has become a subsidiary of Apple Inc [2].

At the bottom of this post, the writer puts a link for you to download FingKinector.

FingKinector works in 2 options:

1. Single run
2. Sequential run

The following image shows the first mode (single run). It is known from [3] that the motor is allowed to be tilted to an angle between -27° and 27°.

Image 2: Single Run mode on FingKinector

The second mode, as seen in Image 3 below - the sequential run, uses a factor to divide the range [-27°,27°] up to 10 sectors. By default this mode will first turn the Kinect to the position of 0°, then starts the sequential movement from either top (at 27°) or bottom (at -27°).

Image 3: Sequential mode on FingKinector

The writer also took a liberty to record an amateur video to demonstrate the sequential mode. The Kinect here is seen with an extra hardware called Kinect Zoom Wide-Angle Lens by Nyko [4]. Its cost is around € 14.00.

Sources:

[1]: Kinect Adapter in Amazon.de

[2]: PrimeSense (Wikipedia)

[3]: Kinect Sensor - Microsoft Developer Network

[4]: Kinect Zoom Wide-Angle Lense by Nyko (Amazon.de)

REMINDER / WARNING:

1. For Windows only. (Linux version will be made available in the next development)
2. The writer is NOT liable for any damage.
3. Use at your own risk.

Have fun!

Download link:
Download link for FingKinector (Windows)