Learning Deep Social Interactions to Identify Positive Classroom Climate

Gardias, Przemyslaw M.

Etd

Learning Deep Social Interactions to Identify Positive Classroom Climate

Öffentlich

Recent work on automatically estimating the level of Positive Climate (PC) in school classrooms, as defined by the Classroom Assessment Scoring System, has demonstrated success in using deep neural networks to model the scene of a classroom as a social network graph. We theorize that by tracking participants within a social graph over time, we can attain higher CLASS prediction accuracy compared to previous work which ignored students’ identities. In this thesis, we (1) propose a process for constructing an ordered social network graph data structure over time. We then (2) conduct two experiments on simulated classroom observations to evaluate the effect of tracking people in order to utilize interactions between individuals when fitting a Graph Neural Network (GNN). Our findings suggest an improvement in classification accuracy when harnessing the feature interactions using the proposed tracking-based approach. Next, in an effort to improve the accuracy of tracking faces over time, we (3) analyze the latent embedding space of pre-trained face embedding networks and find suboptimal discriminability of faces in real-world classroom videos with highly non-frontal pose and very young children. Finally, with the aim of improving the discriminability of these embedding models, we (4) explore the viability of fine-tuning a pre-trained face embedding network on classroom videos, where the labels are extracted in a self-supervised manner. Experiments on classroom videos from YouTube and the UVA Toddler dataset suggest this can be effective: fine-tuning the pre-trained FaceNet, we adjust the embedding network to be better suited for a classroom setting, improving from a test ROC AUC (distinguishing same vs. different face) of 0.95 to 0.98 on unseen classroom observation videos.

Creator