Study of structure disambiguation using eye tracking
Many studies have been done to understand the interplay of utterance
and visual cues in comprehension, disambiguation of the sentence
structure and thematic role assignment. An interesting experiment done
by Tanenhause at el. used ambiguous instructions such as “Put the apple on the towel in the box” the
phrase “on the towel” is a modifier of the noun phrase. However, it
can also be viewed temporarily as a modifier of the verb and can be
used to identify the destination of the apple as the towel. When the
sentence was uttered and an image of an apple kept on a towel and an
empty towel was shown to the subject, the subject initially looked the
apple, but fixated immediately on the empty towel on comprehending the
phrase, “on the towel” as the destination instead of a noun phrase
modifier. However, when the subject was shown two apples one of
which was kept on a towel and the other one on the table, the subject’s
gaze kept oscillating between the two apples while listening to the
noun phrase before fixating on the apple on the towel while listening
to the noun phrase modifier.
Through this project I wish to observe and make plots of proportion
of eye gaze with respect to time for ambiguous sentences in Hindi. For
example, the following sentences are uttered to the subject while he is
shown the pic given below:
राजकुमारी ने सेनापति को तलवार से मारा I
राजकुमारी ने चित्रकार से चित्र बनवाते हुए सेनापति को तलवार से मारा I
अर्जुन कर्ण से लड़ा I
अर्जुन को कृष्ण से ज्ञान मिलने पर वह कर्ण से लड़ा I
The experiment examines whether the attention towards utterance is
closely time locked with the scene information used for structural
disambiguation and thematic role identification.
References:
Pia Knoeferle, Matthew W. Crocker: The Coordinated Interplay of Scene, Utterance,
and World Knowledge: Evidence From Eye Tracking
Paola Merlo, Suzanne Stevenson: Language acquisition and Ambiguity Resolution: The role of frequency distributions