Harvey P. Weingarten – Assessing critical skills for life and work: duelling anecdotes don’t measure up

With summer’s end, more than a half million students are back at it in Ontario’s public colleges and universities, typically investing two to five years and thousands of dollars in everything from tuition fees to technology. For its part, the Ontario government contributes more than $5.5 billion to the public postsecondary enterprise.

Given the magnitude of this public and private investment, we should know whether Ontario postsecondary students acquire the knowledge and skills they need to succeed in life and work. Regrettably, we don’t know – because the system and its institutions don’t measure those skills.

Not that we don’t measure. Student transcripts are teeming with measures of disciplinary knowledge of dates, concepts and formulas. This is typically what professors teach, what they evaluate and subsequently credential. But a postsecondary education is more than just knowledge, particularly in a world where information’s accessibility is impeded only by its shelf life.

Are we measuring what matters? We expect postsecondary graduates to be literate and numerate but recent and pending reports from the Higher Education Quality Council of Ontario (HEQCO) indicate that we’re not assessing basic literacy and numeracy in our postsecondary students – neither when they begin their studies nor when they graduate. Given that appropriate levels of literacy and numeracy are fundamental outcomes of postsecondary education, colleges and universities should be evaluating these skills in all of their students as part of a comprehensive assessment of desired learning outcomes.

We also expect postsecondary graduates to have acquired critical thinking and problem-solving skills. Cue the chorus proclaiming that the mere completion of a postsecondary program verifies the presence of these skills. Evidence-free assertions aside, measuring these skills is a growing focus of higher education research. HEQCO is working with a consortium of Ontario colleges (Durham, Humber and George Brown) and universities (Toronto, Guelph, and Queens) on developing reliable and valid measurements of critical thinking in their students. The results could well be scalable but the Ontario system should be much more active in systematic measurement of cognitive skills. Doing so would position it as among the best public postsecondary systems in the world.

Then there are those skills that could be highly predictive of employment success – the very skills employers identify as most deficient in prospective employees. Behavioural attributes such as determination, persistence, resilience, creativity and entrepreneurship are also largely uncharted territory in higher education assessment. For example, HEQCO research found a proliferation of college and university programs that purport to teach entrepreneurship, yet almost none of these programs actually measures whether graduating students are more entrepreneurial or are more likely to demonstrate these skills in the workplace. There is simply no evidence – as evidenced by a recent Gallup US study which found that 85% of university administrators say their graduates have the skills and competencies to succeed in the workplace but only 15% of employers agree. There is every reason to believe that a similar dichotomy exists in Canada.

Despite considerable consensus on the range of knowledge and skills students should acquire in their postsecondary education, our commitment to assessment doesn’t measure up. We need to do a better job of measuring whether these skills and knowledge are, in fact, taught and learned in Ontario’s colleges and universities – not as an exercise in ranking but as a process for improvement.  Where measurement instruments are lacking, we should advance the research to develop useful measurement tools.

The quality of education provided by the public postsecondary system is simply too important to the future of Canada to leave to debate by duelling anecdotes. We need evidence and that starts with better measurement of what our students are learning and what skills they are acquiring in their postsecondary education.

Harvey P. Weingarten is the president and CEO of the Higher Education Quality Council of Ontario (HEQCO). This post originally appeared in the Globe and Mail.