Reference no: EM132851902

Visual Intelligence

Assignment - Practical work and technical report

Assignment description

The purpose of this assignment is to implement an object recognition method in Python or C/C++ to recognise different objects. Please consult your lecture notes for an overview of object recognition approaches, as well material from the literature. Methods may include geometric (shape) analysis and appearance-based methods. You are provided with a framework to install and use which provides access to Kinect data (see below). You may wish to use OpenCV (opencv.org), a library of programming functions, for real time computer vision. However, bear in mind that your own successful implementation of recognition will attract more marks than using library functions alone (although you may wish to use some of the OpenCV functions to support your implementation.)

The basic requirement is to be able to recognise different objects, presented as streamed depth data from a Microsoft Kinect sensor. To give you an idea of what is expected, consult the following video. You may optionally use RGB (colour) information in your object recognition process.

Note: you are not expected to generate a program that learns objects on the fly. All objects can be learnt offline. Your overall solution should include (1) implementation of an appropriate recognition algorithm; (2) performance evaluation of the recognition algorithm on the test dataset using a confusion matrix.

Project and dataset: You have been provided with:

A CMake C/C++ project (with Visual Studio example) and a Python project, which enable reading of Kinect data (depth, RGB, accelerometer) from a named directory.

Two datasets, one (labelled) for training, and one for testing

To generate the training/test datasets Depth, RGB and accelerometer data has been dumped from a real Kinect (presented with different objects) into a directory that can be loaded by Fakenect. Fakenect is a library which provides an interface to the data without requiring a real physical Kinect. It is not expected that accelerometer data will be used; Depth data should be your primary source of information for recognition.

Note: you are encouraged to use RGB data in addition (you will need to think about calibration and how to register the RGB image to depth) which will attract more marks

Undertaking the assignment: With support provided through the scheduled online interactive session and instructions on Blackboard, install the provided C/C++ or Python project on a lab PC (remotely) or on your own machine. Check that the simple demo of loading and getting access to Kinect data works. Download the datasets provided for the assignment (link to request data is on Blackboard). Start with depth data only and extract frames from the provided training data corresponding to each object, applying a suitably chosen threshold to the depth data. The extracted frames constitute your training dataset. Apply a similar process to the provided test data to generate your test dataset. Extract one or more selected features (e.g. shape and/or colour, based on what you have been taught, outside reading, use of OpenCV, etc.) from your training dataset and build a classifier. Evaluate your classifier on the test dataset and generate a confusion matrix.

Attachment:- Visual Intelligence.rar