Technische Fakultät - AG Wissensbasierte Systeme

Situated Communication in Virtual and Natural Environments

Ipke Wachsmuth

Faculty of Technology
University of Bielefeld

ipke@techfak.uni-bielefeld.de

The general context for the work reported here is human-machine communication in task environments observable by the two communicating partners. Natural communication among human partners to a great extent grounds on their current perception of the situation in their task environment, i.e., is situated. If we want artificial systems to make better profit of the ways humans communicate, they need to be able to process user instructions by integrating what is said with what is seen, and they must be able to maintain knowledge about objects undergoing change in the course of a situation. For instance, if we want to instruct an artificial agent to change a configuration of objects in a dynamic world, then the agent needs to exploit the actual situation, as a source of information, for example, integrate visual input while processing a natural language instruction. We shall give a brief overwiew of some current research projects at the University of Bielefeld, Germany, and concretize these ideas in two examples. In the research focus theme "Artificial Intelligence and Computer Graphics", the VIENA Project ("Virtual Environments and Agents") is concerned with interface agents supporting situated communication with a computer graphics design environment. In a more detailled second example, we present work from the CODY ("Concept Dynamics") Project which is part of the Collaborative Research Centre SFB 360 "Situated Artificial Communicators". Our overall goal is the construction of an artificial system - on the long range an assembly robot - which is able to communicate and cooperate with a human partner in the accomplishment of assembly tasks. In our reference situation, which touches all the ten projects in SFB 360, complex aggregates, such as a model airplane, are to be assembled from the components of a wooden construction kit. The CODY project is concerned with the development of knowledge representations capable of dynamically conceptualizing the situation in the task environment. These integrate informations not only describing the types of the objects involved, but also their changing functional roles when becoming part of larger assembly groups. As representations of individuals are grounded in the the task environment, they can access the spatial knowledge contained therein which thus needs not explicitly be represented.

talk given: Aalborg DK 2-05-96, Manchester UK 18-05-96