Automated Detection of Joint Attention and Mutual Gaze in Free Play Parent-Child Interactions

Abstract

Observing a child’s interaction with their parents can provide us with important information about the child’s cognitive development. Nonverbal cues such as joint attention and mutual gaze can indicate a child’s engagement, and have diagnostic value. Since manual coding of gaze events during child-parent interactions is time-consuming and error-prone, there is a need for automatic assessment tools, capable of working with camera recordings without specialized eye-tracking equipment. There are few studies in this setting, and accessing naturalistic parent-child videos is difficult. In this paper, we investigate the feasibility of detecting joint attention and mutual gaze in videos. We test approach on challenging data of a child and a parent engaged in free play. By combining multiple off-the-shelf approaches, we manage to create a system that does not require much labeling and is flexible to use for view-independent interaction analysis.