Causal Analysis for Visual Gesture Understanding

Matthew Brand and Irfan Essa

Appears in Proceedings of AAAI Fall '95 Symposium on Computational Models for Integrating Language and Vision

We are exploring the use of high-level knowledge about bodies in the visual understanding of gesture. Our hypothesis is that many gestures are metaphorically derived from the motor programs of our everyday interactions with objects and people. For example, many dismissive gestures look like an imaginary object is being brushed or tossed away. At the discourse level, this implicit mass represents a referent in the conversation; at the scene-formation level, the dismissive gesture obeys many of the kinematic and dynamic constraints that would shape an actual tossing. Thus this metaphor provides us with constraints for both discourse annotation and visual processing. In this paper we present some preliminary results interpreting complex gesture sequences in video.

Also available as MIT Media Laboratory, Vision and Modeling Group Technical Report # 327

Here, can do any of the following to get my Tech-Reports:
View them via a Postscript Viewer
Request to download the Tech Report
Send email to request a hardcopy (Please try downloading it yourself first)

Matthew Brand and Irfan Essa
TR#325: Causal Analysis for Visual Gesture Understanding

Irfan Essa,
Last modified: Thu Feb 15 10:06:29 EST 1996