When I started working at the Higher Education Quality Council of Ontario (HEQCO) I asked the obvious question, given the title of the organization: What is quality? I became even more invested in this question when I discovered that the legislation that created HEQCO required us to advise the government on the quality of Ontario’s postsecondary system.
So I did what I was trained to do. I consulted presumed experts. I read volumes of material on the subject. I held long and probing discussions with my peers. For all my efforts, I got no clarity. There appeared to be no consensus on a definition of quality. More discouragingly, some argued that whatever quality was, it could not be measured. The general view was that quality referred to some threshold, adequate or superior level of performance. But there was no agreement on the dimensions of performance, how performance could be measured, or what constituted threshold, adequate or superior performance (and certainly no agreement on what constituted “excellence” or “world class”). And, anyway, in a world of differentiated postsecondary systems, it did not seem reasonable that all institutions would have the same set of performance measures, indicators or levels, even if these could be articulated.
I thought to myself — life shouldn’t be so complicated and, more importantly, if we were to do anything meaningful in the quality domain, surely we would have to have a more useful definition.
Luckily, I discovered other people and organizations in the world that think about quality. The International Organization for Standardization (ISO) is a non-governmental body that develops international standards. Essentially, its definition of quality is the degree to which some product, service or process fulfills requirements or needs. Simply put, high quality results when all needs and requirements are achieved. Low quality results when they are not.
Think of buying a car. Joe wants a car that is great on mileage, small, low maintenance and does not cost very much. Jane wants a car that has a living-room ride, is spacious, screams luxury and prestige, and cost is irrelevant. Joe buys a Toyota Corolla. Jane buys a Jaguar XJ. As long as both Joe and Jane are satisfied that the car they purchased meets their needs and requirements, they both conclude they got a high quality product.
Quality in postsecondary education is no more complicated a concept. Students attend postsecondary institutions and the public and governments invest in them because they expect these organizations to fill a requirement or need. If students get what they need or want from their postsecondary institution, then they got a high quality experience. If governments and the public get what they need or require, then they conclude that they are funding quality institutions.
This definition of quality raises an important and fundamental question — what do we need or require from our postsecondary institutions? Student surveys tell us that the dominant thing students want is a credential to get a good job. But this is not all they want and surely there is more to postsecondary education than just job training. Governments appear to want an accessible postsecondary system that is affordable and produces job-ready graduates. But governments also talk about the role of postsecondary education in economic development, innovation and civil society.
Among all of these things, what we remind ourselves continually at HEQCO, is that colleges and universities are academic institutions. So, if nothing else, a central and non-negotiable need and requirement from postsecondary systems is an education that prepares graduates for success. So what is the core of quality? Making sure that our postsecondary institutions offer an education so students acquire the knowledge, skills and competencies they need to lead successful lives.
We have written previously about the qualities, knowledge, skills and competencies typically expected of a postsecondary graduate. This includes core knowledge in their discipline; basic cognitive skills such as literacy and numeracy; higher order cognitive skills such as problem solving, critical thinking and an ability to communicate; and transferable life skills such as determination, persistence and resilience.
How does one determine quality in a postsecondary institution or system? Very simple. One determines whether the college or university graduates students with the knowledge, skills and competencies the institution says the students need and will acquire if they attend that institution. High quality is achieved if graduates have the advertised attributes. Low quality is when they do not.
Here’s a concrete example: In 2013, Max Blouw, president of Wilfrid Laurier University, wrote in the Globe and Mail that “when a university graduate [of an Ontario university] is recruited, the employer has in their hire an exceptional communicator, an adept researcher, a problem solver and a critical thinker.” This is what is advertised and, frankly, what respectable postsecondary leaders say students need and will get at their institution. What is quality? The degree to which these advertised skills and attributes are achieved.
A lot of measurement questions remain. Next week my blog will report on an international gathering HEQCO is sponsoring in May on how to best measure academic quality.
Thanks for reading.
9 replies on “Harvey P. Weingarten — What is Academic Quality?”
Hello Harvey, good synopsis of a really complicated topic. One observation though – the quality movement always talks about the voice of the customer – quality is determined by the degree to which a product or service meets the needs of the customer. So the key is knowing who the customer is, and what they want/need – and herein lies one of the problems. Every institution talks about the student as the customer – I would argue that the customer for education is in fact the society and the economy. At risk of raising the ire of the educators, let me propose a different paradigm: the school is the factory, the student is the product to which the factory is adding value, and the society/economy is the ultimate customer. Quality needs to be measured primarily from the point of view of the end customer, rather than from the point of view of the product. The difficulty I think comes from the fact that the product in this case is a sentient being who is paying some or all of the costs incurred for the privilege of having value added to them by the institution. Can’t ignore the desire of the student, but also can’t ignore the fact that an extraordinary amount of public money and trust is being given to the institutions to create economically viable citizens. And not meeting the need of the REAL customer erodes trust in the institution.
Jeff,
Your industrialised, converyor belt model of education has raised the ire of this educator/student/ administrator. “Economically viable citizens?” Students are now “products?”
According to your paradigm (not a novel paradigm, by the way), the pedagogical needs of our students are to be framed/determined by the needs of the “REAL” customers (i.e. society/economy)? Sounds a bit abstract… are you, perhaps, referring to the “market?” Would that include local economies? National? Or global markets? And what portion of society (white, male, perhaps? [full disclosure: I’m white AND male]) would decide what the “REAL” customer wants/needs?
I suggest we stop using neoliberal, managerialist goobledegook (customer, product, conveyor belt delivery model, cookie cutter learning outcomes) when talking about education, and return students and educators front and centre of the conversation…
Dear Harvey,
You ask, “So what is the core of quality?” And answer, “Making sure that our postsecondary institutions offer an education so students acquire the knowledge, skills and competencies they need to lead successful lives. ”
When asking complex questions, it behooves you to provide answers that respect such complexity. For example: how do you define “successful lives?”
Thanks for your efforts.
Shawn
In the “teaser” that got me to open the larger post, Harvey Weingarten raises the further question of what “our society requires from postsecondary institutions.” in the main body of the post, however, the word “society” appears only once and in a different context. Instead, for society, he substitutes talk about “an education so students acquire the knowledge, skills and competencies they need to lead successful lives.”
“Society” is thus replaced by notions of individual “success” and this, in turn, is said to be the consequence of having acquired certain “knowledge, skills and competencies” that, no doubt, will lead to satisfactory economic outcomes in the form of measures of employment rates and personal income. “Education,” then, is seen implicitly (cf. the frequency of the term “invest”) and explicitly as an economic investment by taxpayers and customers (formerly known as students) in a system that will meet labour force requirements and personal needs.
But what if our economy (or, more accurately, our political economy) is not itself designed to accommodate individual success? What if our dominant institutions are generating only a “gig economy,” a life of endless employment insecurity and precarious income? What if our corporate structures are engaged in the massive transfer of wealth from the working and middling classes to a tiny plutocracy or a kleptocracy or even a kakistocracy?
And what if our hegemonic social arrangements are moving us inexorably toward a state of permanent war (or preparation for war), a global democratic deficit, and impending catastrophic environmental degradation?
And, finally, what if our so-called “academic” institutions are reinventing and “rebranded” themselves as cheap corporate training facilities and research and development institutions interested mainly in disseminating “employability skills,” commercialized research and hubs of innovation and entrepeneurship in which critical analysis is reduced to a desiccated pedagogy of the Rubik’s Cube in which “thinking outside the box” is merely a way to encourage fixing the box?
I would argue that all of the previous “what ifs” are already fully or largely the “reality” and that what we deem to be success is actually a program for personal and social failure. Meanwhile, our curricula, pedagogy and evaluative techniques (multiple choice questions where essay-type answers are truly needed) do not even meet the expressed desires of potential employers who increasingly loudly lament the inability of “job-ready” graduates to speak and write competently (never mind effectively), to understand the rudiments of their social and cultural surroundings, and to display basic thinking skills beyond those present in an intellectual Skinner box. And, to top it all off, we see around us relentless demands for more STEM courses and the further reduction (qualitatively and quantitatively) in courses in the Humanities and Social Sciences that address precisely the problems that are not just the preoccupations of “social justice warriors,” but even “captains of industry” who understand that something is profoundly amiss, but haven’t quite clued in to what it is.
A good discussion of defining quality depending on the expected return on investment of the stakeholder whether it be government for their institutional investment, the institution itself for enrollment and graduation rates and the learner for the pathway to worthwhile employment. It’s definition therefore varies greatly depending on the expectation.
All else being equal, every person who buys the same model and make of car should expect the same experience. When you put the key in the ignition, it starts. When you press the gas, it goes. When you hit the brakes, it stops. And really from one model to the next, the differentiation may be in the experience, smoothness, performance, but generally, each customer gets the same thing.
However, an education is a different matter. If it were as simple as the car, all students sitting in a classroom with the same instruction and texts and the same hours of study would achieve the exact same grades and results. And yet… they don’t. There are some things we all acknowledge we cannot learn. We do not all expect that if we all take the same classical voice class we all come out virtuoso’s. We do not all expect that if we all have the same salsa class, we all come out ready to dance on stage with the best. We all understand intuitively, that there are innate skills and talents that we bring with us and which education and training enhance but cannot create from nothing. So if we acknowledge this, could the same not be said of math or science or engineering?
That being the case, how can we treat the delivery and experience of education as a product in the same way we do cars? Or students as customers who can all expect to succeed in the exact same manner if they take the exact same program of classes?
That to me is the fundamental flow in this type of measurement of quality. Quality to me, in education, needs to be measured more so in terms of inputs than outputs. Inputs with respect to instructors who are all qualified and trained in pedagogy, curriculums that are thoughtfully planned out, assessments that assess the right things and then, of course, students who are prepared to study. It may be the only way because, in terms of outputs, I’m a complicated human being and many things act upon my ability to learn and succeed.
Harvey P. Weingarten suggests that “High quality is achieved if graduates have the advertised attributes”. However, I suggest that the graduates need to be measured against an external standard, not just what the university advertises as attributes. In “Digital Teaching for Higher Education” I provide examples of aligning education with an existing internationally agreed skill set for “Innovation” and defining a new internationally agreed skill set for “sustainability”: http://www.tomw.net.au/digital_teaching/
Defining “quality” in terms of client outcomes is risky, in that it may adopt too narrow a view of just WHO the clients of higher education are. We assume that student clients benefit by acquiring the requisite knowledge to achieve something in their lives that provides what they want it to provide. We don’t expect all students to become faculty or activists for their discipline, but at least part of that requisite knowledge is recency and some adherence to state of the art in the chosen discipline. In other words, a quality education should be at least a *little* forward-thinking, and not simply a summary of what the student might have acquired 10 or 20 years ago. It’s like a little package a parent gives a child for a journey, to only be unwrapped later in the journey, and satisfy whatever needs the traveller didn’t anticipate, but the parent did. That is, after all, part of why faculty search committees look for faculty on the forefront of their respective disciplines – so that students can have tall shoulders to stand on, and acquire knowledge that comes into blossom down the line. Quality in education is prospective, not retrospective.
Can students know a priori, at the time, or even 5 years later, how tall those shoulders were? Are they capable of being aware of the quality of the education they received? I don’t mean to be condescending here. Rather, there are many things we cannot evaluate the importance of until somewhat later in our journey, like the seemingly banal objects one collected in earlier levels of an adventure game that become critical in later levels. A quality education, then, is not just something that satisfies current needs and wants. It is something that makes the learner smack their foreheads, or tilt their head like the RCA Victor puppy, well after the training period has ended, that facilitates the sort of curiosity that leads to new knowledge, for the individual, for the discipline, for the society that engineered and supported the opportunity.
When talking about academic quality we tend to focus on the academic enterprise hence the divergent views on who the customer is. If we were to view our HEIs holistically, being a system composed of subsystems/units working in synergy towards the attainment of institutional mission and vision, goals and objectives, another set of customers emerges from interdepartmental interrelations and inter dependencies wherein the capacities and capabilities of the other enhance those of the other and by doing so improving inputs, processes and outputs of subsystems and thus those of the system as a whole. By this, I’m referring to the support sector within the HEIs, which is normally left out in an attempt to respond to the question of quality in higher education. For an example the service rendered by HR to the academic enterprise is equally important as the service of ICTs to HR and student support, same as that of LIS to both staff and students, and the alignment of resource allocation, planning and quality management…the list goes on. In addressing the notion of quality in HEIs, the definition of customers within the system is to be widened and broadened to allow for a more inclusive approach to quality.