How to Detect and Track Object With OpenCV

Processing and understanding objects by robots is a complex process designed to produce information based on visual systems and software. Based on idea to duplicate the human vision ability, a computer vision system use electronic parts and algorithms instead eyes and brain. Open Source Computer Vision Library (OpenCV) is the most used libraries in robotics for detection and understanding the objects captured by image sensors.

OpenCV is an open-source library opened for everyone who wants to add new functionalities. It can be downloaded and installed on Ubuntu, Windows or MacOS operating systems. Installation guide with steps and setup is available here.

OpenCV is compatible with next compilers:

• Ubuntu: GCC 4.4.3 (Ubuntu 10.04), GCC 4.6 (Ubuntu 11.10), GCC 4.6.3(Ubuntu 12.04);
• Windows: MSVC 2008, 2010, MinGW 4.5.1 x64, 4.6 x86;
• MacOS: GCC 4.2.1 ;

In order to get started using OpenCV, I made an overview of tutorials and resources focused on OpenCV library.

Detect and Track Objects With OpenCV

In the following, I made an overview of tutorials and guides to getting strted how to use OpenCV for detection and tracking objects. OpenCV is a library for computer visions designed for analyze, process, and understand the objects from images aiming to produce information.

• OpenCV Tutorials – comprehensive list with basic OpenCV tutorials and source code based on the OpenCV library;
• Object Detection & Tracking Using Color – example of application where OpenCV is used to detect objects based on color differences;
• Face Detection Using OpenCV - guide how to use OpenCV to detect one or more faces from the same image;
• SURF in OpenCV – tutorial how to use SURF algorithm designed to detect key-points and descriptors in images;
• Introduction to Face Detection and Face Recognition – face detection and recognition are two of the most common applications in computer vision from robotics, and this tutorial present the steps how a face is detected and recognized from images;
• Find Objects with a Webcam – using a simple webcam mounted on a robot and the Simple Qt interface designed to work with OpenCV, as you can see in this tutorial any object can be detected and tracked in images;
• Features 2D + Homography to Find a Known Object – tutorial with programming code and explanation in order to use two important functions included in OpenCV. These two functions – findHomography and perspectiveTransform – are used to find objects in images. The findHomography is a function based on a technique called Key-point Matching, while the perspectiveTransform is an advanced class capable of mapping the points from an image;
• Back Projection – tutorial based on calcBackProject function designed to calculate the back project of the histogram;
• Tracking Colored Objects in OpenCV – tutorial for detection and tracking the colored objects from images using the OpenCV library;
• OpenCV Tutorials – Based on “Learning OpenCV – Computer Vision with the OpenCV Library” – in order to be familiar with computer vision concepts, these tutorials can be useful for beginner and advanced users to start building applications or to improve the skills;

Tutorials for Detecting and Tracking Objects with Mobile Devices

Mobile devices such as smartphones and tablets with iOS or Android operating systems can be integrated into robots and used to detect and track objects. Below is an overview of tutorials with comprehensive information for tracking objects using different mobile devices.

• OpenCV Tutorial – comprehensive tutorial from where you can learn how to run the OpenCV on iPhone to process frames from video images;
• A Complete iOS OpenCV Sample Project – this tutorial explains how to use the OpenCV library on the iPhone together with the Objective-C to process images. It can be a source of inspiration for robotic projects where an iPhone device is used for control and object detection;
• Using OpenCV on iPhone – tutorial for face detection using OpenCV and iPhone smartphone. From this tutorial you have available all the steps to setup OpenCV as well as programming code;
• Tutorial 1: Object Recognition With OpenCV and Android – Overview of Object Recognition – from this tutorial you can learn how to run the OpenCV library on an Android device and start building application for object tracking and detection in images;
• Developing OpenCV Computer Vision Apps for the Android Platform – resources to detect face using Android device and OpenCV4Android. The OpenCV4Android is a customized library designed to run on Android devices;
• Get Started with OpenCV on Android – a tutorial written by a researcher engineer from Sony that uses an Android device and OpenCV library for face detection and tracking;
• Using the EMGRobotics Robot Controller for Android – tutorial that explains you how to run the EMGRobotics and OpenCV on an Android smartphone to control a robot by face detecting and tracking;

Resources

Below is a list with resources including OpenCV documentation, libraries, and OpenCV compatible tools.

• Welcome to OpenCV Documentation
• JavaCV
• OpenCV4Android SDK
• cvBlobsLib
• Object Detection
• Serialization of cv::Mat objects using Boost

This article was last modified on 05 March 2014.