Harvey P. Weingarten – Learning outcomes: The game changer in higher education

This is the first in a blog series on HEQCO’s key research priorities, in which we share the what, the why and the where-to next.

Focussing on quality, particularly through the lens of learning outcomes, is a game changer in higher education because it influences the way we design, deliver, evaluate and improve academic programs; credential student achievement and design postsecondary systems.

The urgency to focus on quality is palpable. Quality determines the value, reputation and competitiveness of a postsecondary system, institution or program.  Most importantly, the quality of the education they have received determines the perceived quality and competitiveness of our students.

Governments are being held to greater account for the expenditure of public dollars.  Students and the public are questioning whether they are getting value for their investments in higher education.  Employers are concerned about the readiness of graduates for the job market amid a ubiquitous narrative that college and university students are not acquiring skill sets aligned with the demands and requirements of today’s jobs.

Anecdotes abound but the evidence is clear: the most current literacy data from the International Adult Literacy Survey, the Adult Literacy and Life Skills Survey, the Programme for International Student Assessment (PISA) and the Programme for the International Assessment of Adult Competencies (PIAAC) paint a troubling picture of Ontario’s core literacy skills.  As HEQCO reported last fall, students headed to college or university might not have the literacy skills they need to be successful in a highly competitive and increasingly globalized labour market.

Numeracy skills are even more troubling. While PIAAC finds that Canadian literacy skills are near the Organisation for Economic Co-operation and Development (OECD) average, numeracy proficiency is significantly below the OECD average. And the latest PISA survey finds that performance of Canada’s 15-year-olds in math has declined over the last nine years.

Now more than ever, simple assertions or assumptions about the quality of a postsecondary education are no longer sufficient. What is required is a clear demonstration by institutions that their academic programs are adding value and operating at a quality level that serves graduates well professionally and personally.

Ontario has already made considerable progress in identifying learning outcomes. The Ministry of Training, Colleges and Universities (MTCU) was an early adopter of qualification frameworks – a system of expected learning outcomes by credential. Colleges and universities are now using the language of learning outcomes in provincial program standards and Ontario has been an active participant in OECD’s Assessment of Higher Education Learning Outcomes project. The initiative was designed to help determine what final-year university students know and are able to do, and how they compare to students from other countries. HEQCO has also worked with Ontario’s colleges and universities to define and measure learning outcomes in specific disciplines such as the life and health sciences, social sciences and physical sciences.

There is general consensus that the real emphasis now needs to be on assessment of general learning and cognitive skills.  There are the basic core skills: reading, writing, speaking, understanding and manipulating numbers.  Then there are higher order core skills such as problem solving and critical thinking.  There is no consensus yet on how to measure these skills, although we are working on it with a consortium of Ontario’s colleges and universities. Durham College, George Brown College, Humber College, Queen’s University, the University of Guelph and the University of Toronto have committed themselves to the assessment of these core skills, although it could be a year before anything promising emerges and a few more years to determine the scalability to other Ontario postsecondary institutions.

But with Canada’s ongoing participation in PIAAC, we already have consensus on assessing basic literacy and numeracy. Within a year, the Ontario postsecondary education system could capture international leadership by a system-wide commitment to the assessment of basic core skills of students in the system, using the data to improve student performance and institutional quality. Using and/or adapting PIAAC, Ontario’s colleges and universities could evaluate entering and exiting literacy skills in all of their students as the critical first step in a comprehensive assessment of the achievement of desired learning outcomes.

Learning outcomes assessment is an exercise in continuous quality improvement, not punishment — a way of improving education, a way of identifying what is working and where remediation is needed.  By measuring critical learning outcomes, our postsecondary institutions would clearly identify areas for remediation and unambiguously document their value-added impact: the acquisition of these core skills that underlie professional and personal success. The key measure is not the absolute level of these skills; one expects the performance of students in different institutions to vary depending on the institution’s admission policies, target populations and other variables.  The critical measure is the change in the skill sets of students from the time they enter the institution to the time they leave.

A learning outcomes focus will clearly demonstrate the value added of our postsecondary institutions. Importantly, it will also help modernize Ontario’s postsecondary system by shifting the focus from courses and credit hours to knowledge, skills and competencies. It will shift our transfer systems and decisions from arbitrary to evidentiary; our rankings from wealth, selectivity and research intensity to educational value added; our institutional funding from enrolment-based to performance-based.  These changes would position Ontario’s public postsecondary system as one of the most contemporary, progressive and modern higher education systems in the world.

Ontario is an unqualified leader in access to postsecondary education. Quality is the next frontier and Ontario is well positioned to take the lead here, as well. Learning outcomes are now a key part of the lexicon on quality, and quality measures are a key part of MTCU’s new Differentiation Policy Framework.   Ontario is already well on the road, just as students, parents, employers and taxpayers are demanding tangible, clear evidence on the value added of a postsecondary education. The time is now because simply saying that our higher education system is good is simply not good enough anymore.

Thanks for reading.

-Harvey P. Weingarten