K-means Clustering of Student Behavioral Patterns and Advanced Visualization Methods of Learning Technology Data

Norum, Reilly

Student Work

K-means Clustering of Student Behavioral Patterns and Advanced Visualization Methods of Learning Technology Data

Public

This MQP presents two studies that examine middle school students’ mathematical and behavioral patterns as they solve problems in an online algebraic learning game, From Here to There! (FH2T). This game was designed and developed by members of our research team, and our previous studies showed that the game was effective in improving students’ mathematical understanding (Hulse et al. 2019; Ottmar et al, 2015; Chan et al., 2021; Decker-Woodrow et al., under review). Over the past several years, a number of randomized controlled trials have been conducted on this game to test its impact and efficacy in middle school math classrooms. While the larger studies have focused on efficacy, other studies have leveraged the large amounts of secondary data collected from the technology as students solve problems. For example, a recent study using the log data in the game (Lee et al., under review) showed that four subgroups of students emerged based on in-game behavioral patterns, and a subgroup of the students who reattempted the same problem multiple times more often showed the largest learning gains. The first study (Study 1) included in this MQP project provides a replication of this prior study (Lee et al., under review) using a larger and more recent data set (N = 760) that was conducted during the 2020-21 school year. The second study (Study 2) also replicated Lee et al.’s methodological approaches but a different set of variables were used. In both studies, I applied k-means clustering analysis to clickstream data collected in the game and then examined how students’ behavioral patterns varied across the clusters using data visualizations. Specifically, Study 1 aimed to answer the following research questions: Do the same amount of clusters emerge based on students’ behavioral patterns in the game? Do the different clusters show similar changes in their understanding of mathematical equivalence? Do the students’ problem-solving processes and solution strategies show similar variance across clusters? The results of the k-means cluster analysis showed that four distinct subgroups of the students emerged based on four variables of students’ behaviors in the game; the number of problems completed, the proportion of reattempts, resets, and average pause time. Clusters were labeled based on their progress in the game and individually examined to see what type of student made up each cluster. Pretest and posttest scores were included in correlation analysis, showing that both pretest scores and the number of problems completed were positively correlated with posttest scores. Despite not including pretest scores in the cluster analysis, there were distinct differences in pretest scores across the clusters. Study 2 aimed to address the following research questions: How many clusters emerge based on students’ mathematical strategies used in the game? Can the given clusters based on strategies used in the game significantly predict changes in students’ mathematical understanding? How do students’ mathematical understanding vary across the given clusters? The results of the k-means cluster analysis showed that four distinct student profiles emerged based on errors, hint usage, first step efficiency, and first step validity. It was identified that a clear difference between clusters was their help-seeking tendencies (usage of hints). Similar to Study 1, pretest and posttest scores were positively correlated, and there was also a strong positive correlation between pretest scores and the validity of the students’ first step. A linear regression analysis was then conducted to see if posttest scores were significantly explained by cluster results, both before and after controlling for prior knowledge. The results indicated, as expected, the “high knowledge” group performed significantly better on the posttest both before and after controlling for prior knowledge, and the “low knowledge non-help seeking” group performed significantly better on the posttest compared to the control group.

This report represents the work of one or more WPI undergraduate students submitted to the faculty as evidence of completion of a degree requirement. WPI routinely publishes these reports on its website without editorial or peer review.

Creator