Linda Stern and Kim Lam
ED-MEDIA 2007: Proceedings of the World Conference on Educational Multimedia. Hypermedia, and Telecommunications, Association for the Advancement of Computing in Education, 3301-3309, 2006.
We have developed a framework for automated evaluation of open-ended multimedia educational software. Our approach uses a 3-layer architecture for inferring student learning goals from strategies, and for inferring learning strategies from low-level behaviors. We report here on a case study where students used multimedia animation software for learning computer science algorithms. The 3-layer architecture supported the identification of strategies students used in pursuing a goal of intensive learning, as well as supporting the inference of these strategies from a sequence of low-level user actions. We show how the framework supports development of composite models, which can encompass the range of strategies students may use to achieve a single educational goal, and how the framework facilitates incremental building and testing of models of learning. We suggest that these features may be more widely applicable to evaluation of open-ended educational software systems.