Universität Bielefeld - Technische Fakultät - AG Wissensbasierte Systeme

A. Kranstedt, P. Kühnlein & I. Wachsmuth:
Deixis in Multimodal Human Computer Interaction: An Interdisciplinary Approach

In A. Camurri & G. Volpe (eds.): Gesture-Based Communication in Human-Computer Interaction,
International Gesture Workshop 2003, Genua, Italy.
Revised Papers, LNAI 2915, Springer, 2004, pp. 112-123.

Download PDF

BibTeX

Abstract:

This paper presents interdisciplinary work on the use of co-verbal gesture focusing on deixis
in human computer interaction. Empirical investigations, theoretical modeling, and computational
simulations with an anthropomorphic agent are based upon comparable settings and common
representations. Findings pertain to the coordination of verbal and gestural constituents in deictic
utterances. We discovered high variability in the temporal synchronization of such constituents in
task-oriented dialogue, and a solution for the theoretical treatment thereof is presented. With
respect to simulation it is exemplarily shown how the influence of situational characteristics on the
choice of verbal and nonverbal constituents can be accounted for. In particular, this depends on
spatio-temporal relations between speaker and the objects referred to.