Universität Bielefeld - Technische Fakultät - AG Wissensbasierte Systeme

I. Wachsmuth:
Communicative Rhythm in Gesture and Speech.

In A. Braffort et al. (eds.): Gesture-Based Communication in Human-Computer Interaction -
Proceedings International Gesture Workshop (Gif-sur-Yvette, France, March 1999),
Springer-Verlag (LNAI 1739), 1999, 277-289.

Download PDF

BibTeX

Abstract:

Led by the fundamental role that rhythms apparently play in speech and
gestural communication among humans, this study was undertaken to
substantiate a biologically motivated model for synchronizing speech and
gesture input in human computer interaction. Our approach presents a novel
method which conceptualizes a multimodal user interface on the basis of
timed agent systems. We use multiple agents for the purpose of polling
presemantic information from different sensory channels (speech and hand
gestures) and integrating them to multimodal data structures that can be
processed by an application system which is again based on agent systems.
This article motivates and presents technical work which exploits rhythmic
patterns in the development of biologically and cognitively motivated
mediator systems between humans and machines.