(Re-)Examination of Multimodal Augmented Reality

Abstract

The majority of augmented reality (AR) research has been concerned with visual perception, however the move towards multimodality is imminent. At the same time, there is no clear vision of what multimodal AR is. The purpose of this position paper is to consider possible ways of examining AR other than using popular notions and definitions, in order to come to an understanding of multimodal AR. Three concepts are introduced. First, the commonly used terms 'real' and 'virtual' are redefined in terms of stimuli rather than objects, and mediated stimuli are introduced as a new category. This addition facilitates the support of, for example, sensory substitution stimuli in AR. Next, the popular definitions of AR are examined, and a new analysis is presented. Specifically, it is theorized that the core of AR is not necessarily the combination of real and virtual, but the combination of a basis and an augmentation. Such an abstraction allows the re-introduction of augmented virtuality and, more generally, mixed reality. Lastly, a classification system for different forms of multimodal mixed reality that applies this basis-augmentation model is presented.