Etd

Automatic Eye-Gaze Following from 2-D Static Images: Application to Classroom Observation Video Analysis

Public

Downloadable Content

open in viewer

In this work, we develop an end-to-end neural network-based computer vision system to automatically identify where each person within a 2-D image of a school classroom is looking (“gaze following”), as well as who she/he is looking at. Automatic gaze following could help facilitate data-mining of large datasets of classroom observation videos that are collected routinely in schools around the world in order to understand social interactions between teachers and students. Our network is based on the architecture by Recasens, et al. (2015) but is extended to (1) predict not only where, but who the person is looking at; and (2) predict whether each person is looking at a target inside or outside the image. Since our focus is on classroom observation videos, we collect gaze dataset (48,907 gaze annotations over 2,263 classroom images) for students and teachers in classrooms. Results of our experiments indicate that the proposed neural network can estimate the gaze target - either the spatial location or the face of a person - with substantially higher accuracy compared to several baselines.

Creator
Contributors
Degree
Unit
Publisher
Language
  • English
Identifier
  • etd-042318-213325
Keyword
Advisor
Defense date
Year
  • 2018
Date created
  • 2018-04-23
Resource type
Rights statement
License
Last modified
  • 2020-12-27

Relations

In Collection:

Items

Items

Permanent link to this page: https://digital.wpi.edu/show/8g84mm437