Human-In-The-Loop Person Re-Id for Sensitive Datasets

Stelea, Vlad

Etd

Human-In-The-Loop Person Re-Id for Sensitive Datasets

Public

Tracking who appears within videos and when is an important pre-processing step for understanding how students respond to lessons and analyzing their performance within the classroom. However, existing models are often trained on datasets that are not representative of the videos they are applied to leading to them under-performing. For applications where final label accuracy is critical (such as classrooms), we explore how to combine facial recognition predictions with human annotations to reduce annotation actions by a factor of 10. In this thesis we (1) develop an annotation tool and workflow for labeling facial recognition datasets and use it to (2) annotate a custom classroom dataset. We also (3) develop an extensible simulation where we use the annotated dataset to explore the effect of using a (4) pre-trained FaceNet model to generate annotations and (5) retraining this model on newly labeled data. Next we (6) evaluate how the number of exemplars provided for each video affects the overall dataset annotation accuracy. Finally we (7) introduce a new method of quantifying the uncertainty of a model's predictions and explore whether labeling only uncertain examples can be used to reduce annotation actions.

Creator