Leveraging User Testing to Address Learnability Issues for Teachers Using ASSISTments

Bodah, Joshua

Etd

Leveraging User Testing to Address Learnability Issues for Teachers Using ASSISTments

Public

The goal of this thesis is to demonstrate how user testing can be used to identify and remediate learnability issues of a web application. Experimentation revolved around ASSISTments ( www.assistments.org), an intelligent tutoring web application in which teachers create virtual classrooms where they can assign problem sets to their students and gain valuable data which can be used to make informed decisions. Recent log analysis uncovered very low task completion rates for new users on tasks that were intended to be trivial. Suspecting that this could be due to poor user interface design, user tests were conducted to help identify usability problems. Sessions were analyzed, and changes were made between each user test to address issues found. Feedback from user testing led to the implementation of an embedded support system. This support system consisted of a splash page which gave an overview of how the system should be used and a collection of context-sensitive tooltips which tried to give the user instructions on what to do as well as explain various parts of the interface. A randomized control trial was performed to measure the effectiveness of the embedded support. 69 participants were shown one of two interfaces: one with embedded support and one without. Task completion rates were analyzed for each of the groups. We found that the support system was able to influence which links a user clicked. However, although the support system was intended to address poor task completion rates, users in the conditions had similar task completion rates regardless of whether the support system was enabled.

Creator