Harvey P. Weingarten – Assessing critical skills for life and work: duelling anecdotes don’t measure up

With summer’s end, more than a half million students are back at it in Ontario’s public colleges and universities, typically investing two to five years and thousands of dollars in everything from tuition fees to technology. For its part, the Ontario government contributes more than $5.5 billion to the public postsecondary enterprise.

Given the magnitude of this public and private investment, we should know whether Ontario postsecondary students acquire the knowledge and skills they need to succeed in life and work. Regrettably, we don’t know – because the system and its institutions don’t measure those skills.

Not that we don’t measure. Student transcripts are teeming with measures of disciplinary knowledge of dates, concepts and formulas. This is typically what professors teach, what they evaluate and subsequently credential. But a postsecondary education is more than just knowledge, particularly in a world where information’s accessibility is impeded only by its shelf life.

Are we measuring what matters? We expect postsecondary graduates to be literate and numerate but recent and pending reports from the Higher Education Quality Council of Ontario (HEQCO) indicate that we’re not assessing basic literacy and numeracy in our postsecondary students – neither when they begin their studies nor when they graduate. Given that appropriate levels of literacy and numeracy are fundamental outcomes of postsecondary education, colleges and universities should be evaluating these skills in all of their students as part of a comprehensive assessment of desired learning outcomes.

We also expect postsecondary graduates to have acquired critical thinking and problem-solving skills. Cue the chorus proclaiming that the mere completion of a postsecondary program verifies the presence of these skills. Evidence-free assertions aside, measuring these skills is a growing focus of higher education research. HEQCO is working with a consortium of Ontario colleges (Durham, Humber and George Brown) and universities (Toronto, Guelph, and Queens) on developing reliable and valid measurements of critical thinking in their students. The results could well be scalable but the Ontario system should be much more active in systematic measurement of cognitive skills. Doing so would position it as among the best public postsecondary systems in the world.

Then there are those skills that could be highly predictive of employment success – the very skills employers identify as most deficient in prospective employees. Behavioural attributes such as determination, persistence, resilience, creativity and entrepreneurship are also largely uncharted territory in higher education assessment. For example, HEQCO research found a proliferation of college and university programs that purport to teach entrepreneurship, yet almost none of these programs actually measures whether graduating students are more entrepreneurial or are more likely to demonstrate these skills in the workplace. There is simply no evidence – as evidenced by a recent Gallup US study which found that 85% of university administrators say their graduates have the skills and competencies to succeed in the workplace but only 15% of employers agree. There is every reason to believe that a similar dichotomy exists in Canada.

Despite considerable consensus on the range of knowledge and skills students should acquire in their postsecondary education, our commitment to assessment doesn’t measure up. We need to do a better job of measuring whether these skills and knowledge are, in fact, taught and learned in Ontario’s colleges and universities – not as an exercise in ranking but as a process for improvement.  Where measurement instruments are lacking, we should advance the research to develop useful measurement tools.

The quality of education provided by the public postsecondary system is simply too important to the future of Canada to leave to debate by duelling anecdotes. We need evidence and that starts with better measurement of what our students are learning and what skills they are acquiring in their postsecondary education.

Harvey P. Weingarten is the president and CEO of the Higher Education Quality Council of Ontario (HEQCO). This post originally appeared in the Globe and Mail.

3 replies on “Harvey P. Weingarten – Assessing critical skills for life and work: duelling anecdotes don’t measure up”

I appreciate your timely comments. It is true that we expect secondary school graduates to have the literacy and numeracy skills to succeed in their post secondary programs, and that the prerequisites for post secondary programs are a good measure of the skills that students will come with. The College Student Achievement Project, formerly the College Math Project has been reporting a significant gap in achievement in students as they move from secondary school to college studies. To help inform and address the gap in mathematics, specifically and to help bridge the transition, the College Student Achievement Project has developed and is currently beta testing an online mathematics assessment and remediation system which focuses on those critical numeracy skills imperative for success in and beyond secondary school. The earlier field tests of the assessment items with secondary school students has been enlightening and we look forward to examining the results of the current college beta trials of the assessment, as we move toward making this assessment and remediation system available to colleges and college bound students across the province.

Our recent HEQCO research report on Assessment for Improvement of Program-Level Outcomes includes several mini-cases of exemplary institutional practices from the U.S. for assessing program outcomes such as those Harvey has listed. The Transforming Assessment project in Australia and the Assessment Enhancement theme in Scottish higher ed also demonstrate how program outcomes can be approached at a much deeper level than in typical in our Canadian institutions.

Hi Tom:
Is there a link somewhere to this report:
“Assessment for Improvement of Program-Level Outcomes”?
I’m having trouble finding it on the site.

My interest is in the intersection of generic skill assessment with Open Badges, which Mary Catharine Lennon mentioned on a recent trip to Australia.


Leave a Reply

Your email address will not be published. Required fields are marked *