Research Summary:
Prepared by Chris Conway, Queen’s University
The National Survey of Student Engagement (NSSE) is a tool for exploring institutional practices and student behaviours that are known to be associated with good learning outcomes. It is now administered by all Ontario universities as part of their Multi-year Accountability Agreements with the Ministry of Training, Colleges and Universities.
The goal of this final report was the design, implementation, NSSE-based assessment and documentation of engagement-related interventions at several Ontario universities. These were done in order to strengthen the foundation for engagement implementation and assessment practice. The project had four specific objectives, including:
- to establish an inventory of effective intervention “field practices” including those related to data collection, survey administration, intervention design, assessment design and analysis methodology;
- to share (among project participants and more widely) intervention practices and experiences to support improved implementation and assessment efforts in the longer term;
- to conduct formal statistical analyses using best available data and assessment methods to measure the effects of the interventions on both NSSE scores and other key experience and outcome indicators; and
- to inform policy discussions related to the appropriate accountability applications of NSSE.
A variety of intervention projects were assessed at 10 universities in Ontario as part of this report. The key finding was that NSSE item and benchmark measures are generally unable to detect the effects of the relatively modest interventions that were undertaken. The report’s author does not find this to be a criticism of NSSE, or the interventions themselves, but more a comment on the “fit” between NSSE and the scale and scope of the particular interventions. The author suggests that the Classroom Survey of Student Engagement (CLASSE) is a useful tool in assessing course-based interventions.
About the Author
Chris Conway is Director of Institutional Research and Planning at Queen’s University. His research interests include enrolment projection and management methods, performance measurement and accountability and survey research. He received his undergraduate degree at Ryerson, and his master’s degree from York University.