Video Understanding Out of the Frame - an Egocentric Perspective

Dima Damen

University of Bristol, UK

Abstract

The course will introduce egocentric videos as a multi-modal input of direct importance to assistive technology. Footage from wearable cameras is accompanied by additional sources of synchronised modalities (audio, camera motion, gaze), making them a unique source for multi-modal learning. During the course, we will cover common datasets, research questions and progress to date. Particular emphasis will be given to the new trend of operating 'out of the frame' where information is preserved in world coordinate frame, including hand motion, object interactions and object permanence.