Emotional communication with virtual humans

Abstract

In this paper, we present our approach to modelling perceptive 3D virtual characters with emotion and personality. The characters are powered by a dialogue system that consists of a large set of basic interactions between the user and the computer. These interactions are encoded in finite state machines. The system is integrated with an expression recognition system, that tracks a user’s face in real-time and obtains expression data. Also, the system includes a personality and emotion simulator, so that the character responds naturally to both the speech and the facial expressions of the user. The virtual character is represented by a 3D face that performs the speech and facial animation in real-time, together with the appropriate facial expressions.