The application is based on Lucas-Kanade image alignment Algorithm on Raspberry Pi. The APP is a simple implementation of the Inverse Compositional algorithm from the paper "Lucas-Kanade 20 Years On A Unifying Framework". I tried to use the OpenGL ES texture to contain the captured images from the Raspberry Pi Camera and do the calculation partially in GPU.
The basic set-up is shown in above picture, where the Raspberry Pi is connected with a compatible monitor through its HDMI portal. You may also need a USB keyboard and a mouse to operate.
The application is a 2D gyro which can respond to camera horizontal and vertical motions. After running the executable in a terminal, the camera shall be pointed to any relatively constant scene so that the image alignment process would not be suddenly interrupted. The process can achieve the speed at 20 frames per second when we guarantee 10 gradient-descent iterations for every pair of images. The code can be found on: https://github.com/sunjiajing0815/cam_IC_game_V0.1
The camera is plugged in the socket next to the Ethernet portal. After successfully activating your camera and running the project code, we shall see a triangle and rectangle on the top left corner of the screen.
The basic concept is to let the Raspberry Pi recognize the Camera’s displacement through the differences of captured frames.
The algorithm will try to align the adjacent frames as they are almost the same with very little differences. Once the two frames aligned, the shift can be calculated. Therefore, as we move the camera, the triangle will follow. Now we can chase the box as it will change its color every time!
A brief demo on: http://www.youtube.com/watch?v=3eb7Sby7KGc