Refine
Document Type
Language
- English (2)
Has Fulltext
- yes (2)
Is part of the Bibliography
- no (2)
Keywords
- Gestik (1)
- Multimodalität (1)
- Satzellipse (1)
We present an approach to the interpretation of non-sentential utterances like B's utterance in the following mini-dialogue:
A: "Who came to the party?"
B: "Peter."
Such utterances pose several puzzles: they convey 'sentence-type' messages (propositions, questions or request) while being of non-sentential form; and they are constrained both semantically and syntactically by the context. We address these puzzles in our approach which is compositional, since we provide a formal semantics for such fragments independent of their context, and constraint-based because resolution is based on collecting contextual constraints.
This paper addresses the form-meaning relation of multimodal communicative actions by means of a grammar that combines verbal input with hand gestures. Unlike speech, gesture signals are interpretable only through their semantic relation to the synchronous speech content. This relation serves to resolve the incomplete meaning that is revealed by gestural form alone. We demonstrate that by using standard linguistic methods, speech and gesture can be integrated in a constrained way into a single derivation tree which maps to a uniform meaning representation.