Harvey P. Weingarten – Managing for Quality: Classifying Learning Outcomes

My previous contributions on “managing for quality” have addressed the challenges of shifting away from our current policy of managing for access and why learning outcomes is a game changer in the higher education world.   To use learning outcomes productively to improve higher education, and to clear up some of the confusion in the current learning outcomes literature, we need to define the different categories of learning outcomes, and assess our progress and key strategies for each class of learning outcomes.There appear to be four classes of learning outcomes:

Disciplinary content refers to the relevant knowledge and content students are expected to have acquired in their field.  Have they learned the key vocabulary, terms and concepts of their discipline?    Can they do the math their discipline requires?  Have they acquired the critical information in their field?

Basic cognitive skills are fundamental capabilities everyone agrees should be evident in postsecondary graduates.  Literacy and numeracy, the abilities to read and write and to manipulate and use numbers, are the classic baseline cognitive skills.
Higher order cognitive skills are capacities such as problem solving, critical thinking and communication.Transferable life skills (sometimes called “soft skills” or “non-cognitive skills”) are behavioural and personality attributes such as determination, confidence, initiative, persistence, resilience and time management.   They are transferable because they are relevant to success in any job and for both personal and professional success.What do we know about the relationship between postsecondary education and these four classes of learning outcomes?From employer and other reports, postsecondary institutions are doing an excellent job teaching disciplinary content.  For the most part, content is still what most professors teach and most exams evaluate.  In our view, aside from continuing work refining pedagogical and evaluation techniques, postsecondary institutions need little additional motivation, stimulus or help on disciplinary content learning outcomes.

Recent PISA, PIACC and HEQCO reports, suggest that there is room to improve the performance of Canadian students on basic cognitive skills such as literacy and numeracy.  There is international consensus on how to measure these basic skills.  These tests provide a hierarchy of skill levels and indicate how these levels relate to the demands of everyday life. They also provide the opportunity for intra- and international comparisons of skill attainment.  Other than political will, there is nothing preventing us from using the tests now available to measure these important basic cognitive learning outcomes.  We don’t do this now and HEQCO has recommended the measurement of these skills in every entering and graduating postsecondary student in the province for quality improvement purposes.

Other than wide agreement that the development of higher order cognitive skills such as problem solving is a desired learning outcome for postsecondary education, there remains considerable controversy about how to measure these skills.  Some assert that if you graduate from a postsecondary institution you have, by definition, acquired these skills. This assertion is increasingly suspect and there is significant research to develop reliable and valid measures of these skills.  Until these tests are developed, we will be unable to confidently measure these learning outcomes and use this information usefully for quality improvement.  HEQCO is working with a learning outcomes assessment consortium of Ontario colleges and universities to develop such instruments.

Transferable life skills may be the most ignored learning outcomes in postsecondary education, but these are the learning outcomes that employers often identify as the most deficient in postsecondary graduates. Research shows that these skills may be most predictive of employment success.  They are also critical to navigating the decisions individuals will be required to make in their personal lives and may be most instrumental to happiness and life satisfaction.  Ironically, these learning outcomes are measured in early development and school readiness programs but largely disappear from consideration once a student enters formal schooling.  HEQCO is only now beginning to emphasize these important learning outcomes by asking questions such as how can these skills be measured and what pedagogical methods teach these skills.

These are exciting times for learning outcomes in higher education, especially with the growing interest in competency (or outcome)-based education and credentialing.  We will make the best progress when we are clear and disciplined about how we use the term, how we distinguish between different classes of learning outcomes, how we measure outcomes attainment in each of these classes, and how we use that information to improve the quality, relevance and value of postsecondary education.  We find this classification of learning outcomes very helpful to organize our thinking and efforts.  We hope you find it useful and helpful as well.

Thanks for reading.

-Harvey P. Weingarten