Research Summary:
Lessons from the gym: Linking assignments to learning outcomes strengthens core skills
Can learning technology “workouts” both develop and assess cognitive skills? Using a muscle-building analogy, the authors of a new study from the Higher Education Quality Council of Ontario (HEQCO) find evidence that students can strengthen their self-reflective and critical thinking skills with peerScholar – a program developed at U of T Scarborough’s Advanced Learning Technologies Lab.
The study, Taking Learning Outcomes to the Gym: An Assignment-Based Approach to Developing and Assessing Learning Outcomes , suggests that other learning technologies could be developed that simultaneously exercise and assess desired learning outcomes on an assignment by assignment basis — providing a more accurate picture of student achievement.
Project description
An online program that uses peer student feedback in writing, evaluating and reflecting on course assignments, peerScholar’s learning outcomes potential was explored for the study through two experiments, one illustrating the development of self-reflective thought, the second illustrating the development of critical thought. The first experiment in fall 2010 involved 1,292 students in an introduction to psychology course at U of T Scarborough. The second experiment focused on 1,279 students in fall 2011 and winter 2012 offerings of the same course. All participating students agreed to allow their assignment data to be used for research.
In the first, or create-phase, of peerScholar, students submit a digital composition and can be asked to think critically, creatively or reflectively, creating a blank slate for the educator to exercise and assess any specific learning objective. In the assess phase, students log back into the system and review a subset of randomly selected and anonymously presented compositions submitted by their peers. Students score submissions on quality and comment on both a strength and a key area for improvement for each submission. “The process of assessing peer work clearly implicates critical thought and self-reflection,” say the authors. “Students are analyzing, contrasting and evaluating… An assignment could be configured to exercise any other learning objective.”
In the reflect/revise phase, students are assessing written comments about their work. The authors note that many of the learning outcomes exercised in the assess phase are being exercised again, this time in the context of their own work.
Findings
Based on outcomes from specific assignments given to the two experiment groups, the authors found that participation in a peer-assessment assignment enhanced students’ ability to identify and rank quality, and these critical thinking skills increased with each experience. “The suggestion is that critical thought – or at least an index related to it – can both be exercised and assessed during the course of a single assignment.”
The study also found that a single experience of assessing the work of six peers was sufficient to improve students’ ability to assess the quality of their own work, indicating improved self-reflection skills.
Recommendations
The authors say that the approach could be extended to other learning technologies and that similar analyses should be done for other assignment contexts, “creating a broad spectrum of tools both to exercise and assess a range of learning objectives in different ways…But the next step would be linking assignments to learning outcomes…It is time to take that next step, a step into the gym of the mind.”
Authors of Taking Learning Outcomes to the Gym: An Assignment-Based Approach to Developing and Assessing Learning Outcomes are Steve Joordens, Dwayne Pare and Lisa-Marie Collimore, Advanced Learning Technologies Lab, University of Toronto.