Nuances in Efficacy of Computer-Based Learning Platforms: Understanding the Interplay between Instructional Design, Engagement, and Learning

Vanacore, Kirk

Etd

Nuances in Efficacy of Computer-Based Learning Platforms: Understanding the Interplay between Instructional Design, Engagement, and Learning

Public Deposited

Assessing the impact of educational programs on student learning is a vital part of educational and learning sciences research. Randomized control trials are often employed to evaluate the effectiveness of these programs, with results typically presented in binary terms – effective or ineffective. However, programs and practices may be effective for some students and not for others, with the program's impact possibly dependent on specific implementation parameters and students' attributes. One under-explored dimension that likely influences the efficacy of educational programs is student engagement. This dissertation presents research on two opposing engagement behaviors -- gaming the system and productive persistence -- commonly exhibited in computer-based learning platforms. The studies included in this dissertation explore how programs' instructional designs can influence students' behavior and how students' behavioral tendencies moderate the programs' efficacy. For students who game the system, The evidence suggests that the most argent `gamers' may benefit from delayed access to hints and feedback, with mixed evidence on the impact of gamification. Although one gamified CBLP likely negatively impacted `gamers' learning, another had no interaction with students' propensity to game the system, indicating a complex relationship between disengagement and instructional designs. On the other end of the spectrum, gamified performance-based feedback positively affected students' productive persistence behaviors. Furthermore, the impact of various programs differed based on students' tendencies toward engaging in productive persistence. This finding suggests that the ways in which programs allow for and encourage productive persistence may influence whether and how students learn. Overall, this work highlights the need to consider behavior heterogeneity —- how a program's effect varies based on participants' behavioral tendencies -— when evaluating whether and for whom programs are impactful. If a program's impact depends on students having propensities towards or away from specific behaviors, instructional designers should consider how programs can be developed to increase positive learning behaviors and provide students with the educational experiences that will benefit them most.

Creator