It isn’t exactly news that skills matter. A centrepiece of Ontario government policy are recommendations from its Highly Skilled Workforce panel that promote the importance of skills in the province’s future economy. The federal finance minister’s Advisory Council on Economic Growth recently recommended the creation of a national, independent agency — the FutureSkills Lab —dedicated to the development and assessment of the skills of Canada’s workforce. Meanwhile, concern over a real or perceived gap between the skills of Canada’s workers and the demands of the modern labour force is a topic of continuing conversation.
Yet, while so many are talking about skills, almost no one is measuring them. We at HEQCO are.
The postsecondary system in Canada is supposed to be a primary place where students acquire the critical skills they need to succeed in life and in the workplace. But do they? One of the largest research projects currently underway at HEQCO is the Essential Adult Skills Initiative (EASI), which aims to find out. The pilot project uses the Education and Skills Online, a version of the Organization for Economic Cooperation and Development’s PIAAC test, to measure the changes in literacy, numeracy and problem-solving skills in college and university students from the time they begin their programs to the time they graduate.
EASI is also designed to assess the feasibility of skills measurement on a large scale in postsecondary education. The project is critical to achieving one of HEQCO’s long-term goals — to have every Ontario postsecondary institution annually identify, evaluate and publicly report the skills and competencies its students have acquired.
Participation in EASI is voluntary and, so far, 11 Ontario colleges have signed up to take part in the trial and 10 universities have shown interest. In the fall of 2016, we tested incoming students at the 11 colleges and we are currently testing final year students at those same institutions. Testing at universities, of both incoming and graduating students, will get underway in the fall of 2017.
So what do we know so far? We can say that our students’ performance on the test was well within the range that we had expected given comparable data from the PIAAC, giving us confidence about the test’s reliability. The bulk of incoming students performed at a level higher than the PIAAC average for high school graduates in Ontario and Canada between the ages of 16 and 34.
We have also identified and resolved important logistical and methodological issues that needed to be addressed regarding student recruitment, test administration, data management and so on — issues that are important as we consider scaling up the test to a provincial or national level. At this point, we are confident that we have a system and a set of processes in place that allow us to scale up the project to whatever level we wish.
EASI has too many moving parts to be described as an easy project, if you’ll excuse the pun. But we’ve struck a good balance so far. Our early results suggest that we’re using the right assessment tool. We’ve created an efficient system for working closely with our institutional partners to manage testing at multiple sites with diverse needs. And though student recruitment remains a challenge, we’ve expanded best practices from the fall 2016 trial to improve response rates.
We remain enthusiastic and excited about the EASI project, in spite of the considerable resources it consumes. We thank Employment and Social Development Canada and the Ontario Ministry of Advanced Education and Skills Development for funding that has allowed us to handle the unexpectedly large response to the project. We remain convinced, as we have argued before, that measuring the skills of postsecondary students is critical. We will issue more detailed results as the data become available. Stay tuned.
HEQCO’s EASI team includes Greg Moran, Lauren Hudak, Sarah Brumwell and Erin Maloney.