Cognitive Agents in Virtual Worlds: a Middleware Design Approach

Abstract

In the past decade, significant advances have been made in the technologies for creating virtual worlds. These advances are accompanied with the need for intelligent, human-like characters to populate these virtual worlds. Such characters are also known as intelligent virtual agents (IVAs). IVAs are autonomously operating characters that not only visually resemble humans but also exhibit human-like behavioral and social qualities. For instance, they may have certain personality traits, express emotions or are able to engage in social interactions. Or they may be designed to be part of certain social groups, cultures or organizations, influencing the way in which they behave and the decisions they make in different situations. When considering the simulation of an IVA's cognitive behavior, this is the research domain of agent technology and intelligent agents. As an example, a popular software model for developing intelligent agents is the BDI (beliefs-desires-intention) model for simulating human practical reasoning. Also several general cognitive architectures have been proposed that can be employed for modeling human-like, cognitive abilities such as reasoning, planning or learning. In order to make use of such techniques when designing IVAs, one is faced with the problem of integrating cognitive agents within a virtual environment. In this environment they become embodied and ought to sense and act through human-like sensors and actuators. Researchers and industry developers alike have demonstrated such integrations before. However, often ad-hoc approaches are taken in these efforts, leading to solutions which are difficult to transfer to different application domains and software platforms. It is found that there is little fundamental research that focuses specifically on the design issues of integrating agents in virtual environments. In this thesis the aim is to fill this gap. The goal of this thesis is twofold. First the goal is to offer a thorough understanding of the technical and conceptual design challenges for coupling game engine technology and agent technology. On the technical side, this concerns the technical connection between the founded software platforms of game engines on one side and multi-agent systems (MASs) on the other side. On the conceptual side, this concerns the interface design between a cognitive agent on one side and its virtual embodiment on the other side. Here topics will be addressed such as environment perception, behavior planning and realization, behavior and intent recognition, coordination and communication. Second, based on the acquired insights, the goal is to investigate the potential role a general-purpose middleware could play to facilitate designers in realizing a structured coupling between MASs and game engines. On the technical side, roles are proposed to facilitate data and process synchronization between the two systems. Design contracts are established using ontologies. On the conceptual side, roles are proposed to facilitate designers in modeling an agent’s sense-act interface and to facilitate them in designing IVA processes such as perception, behavior planning, coordination or communication. As a proof-of-concept, proposed techniques are described based on the CIGA middleware platform that has been developed.