Toward Improving Effectiveness of Crowdsourced, On-Demand Assistance From Authors in Online Learning Platforms

Haim, Aaron

Etd

Toward Improving Effectiveness of Crowdsourced, On-Demand Assistance From Authors in Online Learning Platforms

Public Deposited

In this experiment, we have a set of authors made up of teachers and undergraduate college students who we paid to write student-supports, which are typically hints and explanations, to be given to students on-demand while solving problems assigned by their teachers in the ASSISTments platform ( https://www.assistments.org). We want to see if we can tell which authors are, on average, producing student-supports that cause better student learning. We conducted a month-long intervention where students were exposed to support from different authors. In this experiment and its replication, we randomized the authors of the student-supports and analyzed a set of pairwise comparisons between authors. We failed to find evidence that we can reliably tell the difference between authors. It could be that our authors produce equally effective student-supports, or it could be that this work was underpowered, and we failed to recruit enough students to discover existing differences. All data and analysis being conducted can be found on the Open Science Foundation website ( https://osf.io/zcbjx).

Creator