U of T rubrics assess student learning in communication , teamwork and design
Researchers at the University of Toronto have developed a set of rubrics to assess student learning in communication , teamwork and design , which can be applied to a wide range of courses and assessment types , according to a new report from the Higher Education Quality Council of Ontario (HEQCO).
Development of Analytic Rubrics for Competency Assessment follows on a three-year project to create rubrics that could be used to assess student learning across various contexts while giving instructors the flexibility to adapt the rubrics to meet their individual needs.
The project , one of the initial seven in HEQCO’s Learning Outcomes Assessment Consortium , sought to develop valid , analytic rubrics to assess learning outcomes in five key competency areas: communication , teamwork , design , problem analysis and investigation. Ultimately , rubrics were developed for the first three , while indicators (specific learning outcomes) were compiled for the latter two.
Project description
A literature review was conducted to create a comprehensive list of learning outcomes in the five competency areas , as well as a list of more specific indicators. Rubric descriptors for design , communication and teamwork were drafted and later modified through consultations with instructors and departmental administrators. The outcomes and indicators for problem analysis and investigation were validated through a systematic Delphi technique and work continues on a set of descriptors for the two. The researchers then tested the design , communication and teamwork rubrics with graduate students , teaching assistants and instructors , incorporating the feedback to produce a set of revised rubrics that can be applied to a wide range of courses and assessment types.
Appendix A contains the final version of the rubrics for communication , teamwork and design while Appendix B contains the indicators list for problem analysis and investigation. Appendix C contains a series of papers produced over the course of the project to help users in developing their own customized rubrics. The rubrics are also available through HEQCO’s Learning Outcomes Resource Room.
Findings
Rubrics are only as good as the training that accompanies them , the authors note , adding that assessors should be trained on how to use the rubric to effectively assess student work. Particularly in the case of teams of instructors evaluating work in the same course , the authors emphasize the importance of a shared understanding of rubric terminology and the quality of work that exemplifies each performance level , leading to greater consistency in grading.
Instructor customization is also important , the authors say. The rubrics were designed so that instructors could , for example , add spaces for comments or items to allow assessors the ability to not only select within , but also between performance levels.
Teamwork skills are particularly difficult to assess , in part because they require observation and interpretation of behaviour. The authors recommend that teamwork be assessed using multiple sources , in particular students’ own reflections , reflections from their teammates and evaluations from external observers.
“Learning outcomes assessment has become an increasingly integral part of educational policy at the postsecondary level ,” the authors say , given its ability to provide concrete information on the quality and effectiveness of higher education. They suggest that the rubric bank be integrated into existing learning management technology to help ensure that the institution as well as its students receive detailed information on student learning and performance.
Authors of Development of Analytic Rubrics for Competency Assessment are Gayle Lesmond , Susan McCahan and David Beach , University of Toronto.