Postdoc job talk - Visual experience through time: Characterising the role of scene semantics and grammatical structures in film and narratives.

Abstract: A great deal of visual cognition research deals with punctate events, short bursts of cognitive effort, and recognition problems with single end point as a desired solution. But much of what the visual system allows us to do involves sustained mental effort, meaningful structures and goals that develop over time, and a sense of narrative that can be related to linguistic-like representations. My research seeks to characterize these attributes of visual processing. In this talk, I am going to present three projects that get at these questions. In the first part of the talk, I am going to use a temporal change blindness paradigm in film to argue that semantic content of the film guides our ability to detect visual disruptions during film viewing. In the second part, I am going to suggest that the organization of the semantic information involves hierarchical grammar. I am going to demonstrate this using visual narratives and computational psycholinguistics tools. The final part of my talk is an attempt to study how we represent events and their underlying contents. I will present some ongoing work involving a storyboard paradigm - where participants are instructed to build a story using still image frames from the watched movie. I am going to show that the chosen image frames can be used to study how we represent events, and conclude with some future directions.