Tracking with Smart Sensors - Autonomous Exploration of Unknown Environment by Fusion of Stereo Vision and Inertial Sensors (FWF - P15748)

In this project, a new, generic approach to real-time tracking using a combination of "smart sensors" is searched. We plan to use a sensor suite consisting of a fixed, calibrated stereo rig together with an "inertio-tracker" based on accelerometers and gyroscopes. These two sensor types provide complimentary characteristics: visual sensing is very accurate at low velocities while inertial sensors can track fast motions but suffer from drift particularly at low velocities. This inside-out tracking system will be subjected to arbitrary motion and shall operate in cluttered multi-object scenes with multiple and independent motion and some static, but 3D background. A typical example might be a person carrying the sensors and walking through a city. The primary goal is a reliable reconstruction of the trajectory of the system itself, as well as the recovery of 3D structure required for successful tracking. Similar to the perceptive capabilities of a human, the system shall operate autonomously, without requiring additional information about its localisation and pose.