Drawing Outside the Lines: Tracking-based Gesture Interaction in Mobile Augmented Entertainment

Abstract

We present a proof-of-concept study for tracking- based gesture interaction in an augmented reality setting using tablets. By tracking a pen in front of a tablet using it’s integrated camera, we are able to map certain motions to gestures, which in turn are used to interact with the application. A comparative user study investigates the feasibility and usefulness of our approach with a simple augmented reality board game allowing translation and drawing gestures to move and create virtual board pieces, respectively. In particular, we demonstrate that users can handle it (and to what degree) and that they enjoy it (and what they potentially dislike). The results from the 25 participants of our experiment provide both subjective and objective evidence of the potential of tracking-based gesture interaction for augmented reality gaming.