Measuring Learning Requires Answering Foundational Questions

A professor has responded to calls for measuring student learning by arguing that the field must first answer a host of foundational questions. The column is a direct engagement with fellow columnist Doug Lederman's recent argument that a failure to measure learning is damaging higher education's public standing.

The author, a writing instructor with nearly two decades of experience, agrees that proving learning is important but contends the conversation is premature. They assert that shifting from 'schooling'-following processes for a grade-to genuine 'learning' has been their professional focus, and they can reliably discern when a student is learning. However, they acknowledge Lederman's point that personal authority and knowing are different from providing demonstrable proof acceptable to external audiences.

The Core Unanswered Questions

Before designing any measurement system, the column poses that higher education must explicitly address several core issues. The first is why winning back public trust is now a necessity when it was previously allowed to slide. The author questions what societal shifts have created this demand and whether addressing those shifts directly might be a better path to restoring confidence.

A critical subsequent question is identifying the primary audience for the measurements. Is it for students, parents, employers, or government bodies? The author suggests that designing a measurement for students might be relatively straightforward, but creating one that satisfies other stakeholders would be far more challenging.

Purpose and Application

The intended use of the measurement is another vital consideration. The column asks whether the data would guide teaching methods, determine funding for programs or institutions, serve as a consumer guide for prospective students, or inform major public policy decisions. Also, it questions whether the goal is to rank students and institutions against each other or to measure individual student progress over time.

Logistical and methodological questions abound. The author asks when and how often measurement should occur: annually, per course, at entry and exit, continuously, or even years after graduation. They also question whether everyone needs to be measured or if a representative sample would suffice, and how precise the measurements need to be.

Defining Evidence and Avoiding Pitfalls

The type of acceptable evidence is another hurdle. Would self-reporting, subjective judgment, or objective testing be considered valid? The column also raises the issue of comparability across disciplines, asking if the same measurement can apply equally to a music performance major and an engineering student.

Perhaps most importantly, the author highlights the risk of Campbell's law, where the act of measurement itself becomes the goal, distorting the very learning it aims to quantify. This phenomenon has historically undermined well-intentioned assessment efforts across all educational levels.

The Institutional Reality

The author expresses skepticism that learning is the current main operational focus of universities. They cite a 2016 statement from Carol Christ, then vice chancellor of UC Berkeley, who said colleges and universities are 'fundamentally in the business of enrolling students for tuition dollars.' The author believes this reality contributes to declining public trust and questions whether a new focus on measuring learning could trigger a fundamental shift in institutional priorities.

They doubt that proof of learning would significantly alter student choices, using Harvard as a hypothetical example. Even if data showed elite institutions were average in build learning growth, the author suspects their prestige would remain largely unaffected. The column ends by hoping that administrators involve frontline educators in these discussions to seek meaningful answers rather than convenient ones.