Surface Gesture & Object Tracking on Tabletop Devices

In this thesis, we are interested in the use of tabletop surfaces for interactive manipulations. We focus on the implementation of Image Processing algorithms and techniques in two projects exploiting a horizontal surface: "Tangram Project" and "MirrorTrack".

The "Tangram Project" studies children's mathematical skills when manipulating geometrical shapes. This project is supported by NFS (NSF 0736151) based on the proposal "Social Organization, Learning Technologies & Discourse: System Features for Facilitating Mathematical Reasoning in PreK-3 Students" by M. Evans, F. Quek, R. Ehrich and J. Wilkins. Our contribution is the design and realization of visio-based tracking software that could be used in a classroom. Our implementation offers three modes of interaction making it easier to study the children's behaviors in specific situations and constraints.

The "MirrorTrack Project" is an idea described in previous research [P.-K. Chung et al,2008a] [P.-K. Chung et al,2008b] using a horizontal surface with two side-mounted cameras to track fingertips. Our contribution to the "MirrorTrack Project" is the design and realization of video-based interaction software. "MirrorTrack Project" provides an improvement to one of the Tangram modes (the Virtual mode) by providing real 3D fingertip location above the surface. Among other functionalities, it provides hovering and touch detection [Y. Verdie et al, 2009].

We conclude by describing the possibility of merging those two systems and by highlighting the benefits of such a fusion. Integrating "MirrorTrack" with the "Tangram project" provides even more interaction opportunities for the children.