Results are promising
HEQCO-led Ontario pilot in learning outcomes part of OECD global project

Ontario is the sole Canadian provincial partner in a global pilot project on learning outcomes initiated by the Organisation for Economic Co-operation and Development (OECD) and according to a preliminary report from OECD, the results are promising.

At the request of the Ontario Ministry of Training, Colleges and Universities, the Higher Education Quality Council of Ontario (HEQCO) joined the OECD’s Assessment of Higher Education Learning Outcomes (AHELO) project in 2011 to help determine if standard generic and discipline-specific tests can be used in different countries to measure what final-year university students know and are able to do. The project is one of three initiatives HEQCO is involved in to define and measure learning outcomes, working with Ontario’s colleges and universities in partnership with international organizations.

For the AHELO project, HEQCO administered a test to final-year students in civil engineering degree programs at Carleton University, McMaster University, University of Ottawa, Queen’s University, Ryerson University, University of Toronto, University of Waterloo, Western University and University of Windsor. The same test was provided to students in Australia, Columbia, Egypt, Japan, Mexico, Russia and the Slovak Republic.

Key among preliminary findings in the OECD report, implementing the test in a similar fashion around the world was possible and the tests could yield generally valid results. The report notes lessons learned in the global initiative and practices that could improve implementation and administration. A final report next spring will include more detailed analysis of the feasibility study results.

“Ontario was pleased to participate in the AHELO feasibility study,” says John Milloy, Minister of Training, Colleges and Universities. “Our government is committed to developing the right tools that measure students’ ability to use and apply knowledge in postsecondary education.”

The Ontario AHELO pilot was conducted in two phases. Initially, small focus groups of civil engineering faculty and students reviewed the test to ensure that it measured skills and knowledge appropriate to the discipline in Ontario. In the second phase last spring, more than 400 Ontario civil engineering students in the final year of their program completed the 90-minute test – representing approximately 60% of the graduating class.  Institutions will receive their individual reports this month and HEQCO will produce a report on the Ontario pilot in the spring.

“Learning outcomes figure prominently in our research plan for the coming year,” says HEQCO president and CEO Harvey Weingarten.  “With a consortium of colleges and universities, we will focus on the measurement of core learning skills such as problem solving, critical thinking, numeracy and oral and written communication -- skills that students would be expected to have mastered by the end of their postsecondary education.”

All three HEQCO learning outcomes projects build on the provincial government’s work in quality assessment and learning outcomes. The Tuning project, to be completed in spring 2013, brought together Ontario faculty members from the social sciences, physical sciences, and life and health sciences to identify and consider ways to assess learning outcomes across diploma, degree and master’s levels.  HEQCO and eight Ontario colleges and universities are also completing a pilot of the Collegiate Learning Assessment, developed by the Council for Aid to Education to test critical and analytical thinking skills at the beginning and end of a degree/diploma program.