Individual and social processes in image description generation

Abstract: Most language use takes place in situated environments where visual perception goes hand in hand with language processing. I will discuss some recent projects by my research group related to modelling language generation in visually grounded contexts using state-of-the-art AI techniques. I will first focus on individual cognitive processes and present a model of image description generation that exploits information from human gaze patterns recorded during language production. In the second part of the talk, I will move on to two-person dialogue setups. I will discuss our recent work on generating referring descriptions that are grounded in the conversational and visual context. I argue that computational models of language generation can help us to better understand the cognitive processes underpinning these abilities in humans, as well as contribute to more robust language technology tools and to user adaptation in dialogue systems.

GoToMeeting info:
https://global.gotomeeting.com/join/765474861

This seminar aims to open that dialogue between the fields of Cognitive Science and Artificial Intelligence. 

More info here