Think Twice, Measure Once
Carpenters are well advised to measure twice and cut once because premature cuts can waste time and valuable materials. Similarly, educators should think twice and measure once because ill-conceived measurements can both distract and distort.
What one chooses to measure generally becomes the focus of time and attention, often to the exclusion of other topics. Thus, if one chooses to measure achievement in literacy and numeracy other subjects such as science, social studies, the Arts and Technology tend to take a back seat, let alone the more subtle issues of social-emotional learning and character development that are critical to a quality education.
The distraction caused by a narrow focus for measurement may also distort outcomes if what is measured does not represent the full array of literacy and numeracy objectives. For example, literacy tests usually include impromptu writing but not planned and edited writing, which is arguably a more important skill. They also generally omit oral communication which is critical for employment, citizenship and life in general.
Although no responsible educator would “teach to the test,” it is virtually guaranteed that the things that are measured and reported will become the focus of attention and that other things will be relegated to the margins. This unintended consequence within schools is compounded in the community when the available data is misused to generate supposed quality indices of limited scope and questionable meaning. A prime example of this disservice is the “Report Card” published by the Fraser Institute in many provinces, which is granted far too much attention and credibility, thus misinforming the public and confusing the important discussion about improving educational quality.
The decision to measure some outcome must be made with full recognition of the potential for distraction and distortion. This is not a reason to forego measurement of outcomes, but it is a reason to employ an array of indicators that span the full range of important outcomes and to ensure responsible, representative reporting and use of the data. Simply proceeding with what is easiest to measure is counterproductive because of ‘collateral damage’ due to the distraction and distortion this causes.
Educational assessment is a complex business and it is expensive to do it well. However, it is also essential. Since it is impossible to imagine funding a census assessment (i.e., all aspects for all students) that adequately represents the full range of important educational outcomes, a sampling approach that delves more deeply by using subtle and sophisticated measures (including qualitative techniques such as case studies and direct classroom observations) is the only credible way to monitor system performance and provide data that is useful for improvement. Unfortunately, while this is the most productive approach for educational purposes, it does not serve the perceived need in some quarters to provide all parents with an independent measure of their child’s learning.
Educational discourse and school improvement clearly requires an evidential foundation, but it is important that the evidence be collected in ways that adequately reflect outcomes. A good assessment program begins with well-understood questions and intentions, which are the foundation for a sophisticated program of evidence gathering, interpretation and constructive use. It does not simply measure what is easiest or draw excessive inferences from what happens to be available.
Meet the Expert(s)
Bruce Beairsto
Retired school superintendent, educational consultant and adjunct professor at Simon Fraser University
Bruce Beairsto is a retired school superintendent, educational consultant and adjunct professor at Simon Fraser University.
Read More
