Who would have ever predicted that a career as an elementary French teacher would eventually lead me to working for an educational publisher as a researcher and change agent? However, as someone who was nearing the completion of her doctoral studies, conducting research as opposed to teaching in the classroom did seem like the next logical step; a practical way to apply my new skills. In November 2013, when Pearson Canada made the bold commitment to ensure that its products and services have a measurable impact on people’s lives through learning, I decided to accept the challenge of leading the company’s internal and external efficacy research agenda nationally.
Working on the corporate side of education has been a big learning curve for me, but it allows me to continue to focus on the learner, which is what drew me to the profession in the first place. Two years in, the more I work with colleagues and leaders in K-12 and higher education, the more I realize how similar they are. Despite different funding and accreditation models; assessment and evaluation practices; ages, skills, and motivation of students; or the structure of the curriculum vs programs, designing products and services needs to start with a clear idea of the intended learner outcomes.
There is an important difference between learner and learning outcomes. The latter are statements that are often found in curriculum documents about what has been learned. A learner outcome is far more expansive – it is the way a human life is transformed by an educational experience. Internally, teams identify learner outcomes using logic models, outcomes workshops, and feedback from customers and learners. The four learner outcome themes apply to both K-12 and higher education settings.
- Learner access and experience involves the learners’ ability to access the content at his or her level and level of satisfaction/engagement with the content. Teachers and instructors focus on this as they get to know their learners’ prior knowledge and interests and then design learning to meet their individual needs.
- Timeliness or completion refers to the progress the learner makes through the program going at his or her usage rhythm to achieve the standard or competence. Educators in both K-12 and higher education contexts achieve this goal through assessing learner progress and providing on-going formative feedback.
- Standard of achievement or level of competence reflects the acquisition of particular concepts, skills or qualifications as a result of using the resource or taking the ‘course.’ Teachers and professors gather evidence using a range of self, peer and teacher assessment and evaluation practices.
- Learner progression is much more longitudinal in nature and may actually extend beyond the learner’s interaction with the Pearson resource at the school or institution. This outcome refers to the ability to progress to the next unit, grade, course of study or training, or even into the workforce.
To ensure that Pearson developers and designers are doing the right things to deliver the learner outcomes they want, team members conduct an efficacy review – an assessment of a product or program against 12 criteria. By giving each criterion a rating on the four-point scale (green to red) we gauge the likely overall effectiveness of a product while identifying specific areas for improved impact. The team then constructs an action plan with detailed roles, responsibilities and timelines for required change. Follow up check-in meetings afford everyone involved in building the product with an opportunity to engage in on-going monitoring and reporting on progress.
Since 2013, we have also had the chance to conduct efficacy reviews with educational leaders and researchers external to Pearson. Results from the 11 different reviews with school districts, colleges and universities across Canada suggest further similarities between K-12 and higher education vis-à-vis local system improvement or implementation efforts. To set improvement and implementation plans up for success, there is the common need to:
- include more voices around the table early on to ensure buy in
- gauge teacher, faculty and user capacity for change by assessing readiness and designing on-going access to differentiated professional learning
- document and communicate the desired outcome for an initiative more broadly so all relevant stakeholders understand the rationale for change
- clearly outline people’s roles, responsibilities, governance, risks and timelines
- collect comprehensive and quality feedback and evidence from all stakeholders
- communicate progress at the institutional, user and implementer level
We are at the beginning of our internal transformation and are thrilled to be collaborating with K-12 and higher education. As our products enter the market and we investigate learner impact, additional points of intersection between school districts, colleges and universities are sure to emerge. The next phase: sharing what we learn with the broader educational community.
Tania Sterling is vice president, efficacy, at Pearson Canada.
Our opinion is that the opinions expressed by our guest bloggers are their opinions, and not necessarily those of HEQCO.