Beware the Bicycle

One of the most difficult situations I faced as a teacher involved a boy and a bicycle. During an individual reading conference, one of my Grade 4 students confided that his mother had promised him a new bike if he got an A in Reading on his report card. I knew from the assessments we’d done that he was sounding out many words without comprehension, and I wanted to refer him for diagnostic assessment with our special education teacher. Putting anything more than a D on his report card would not only misrepresent his achievement, it might also disqualify him from receiving extra help. It would certainly send the wrong message to his parents. But a D didn’t seem fair, and I felt like I was robbing a boy of his bicycle.

Everyone wants classroom assessment to be fair – teachers, leaders, parents and students alike. The challenge for teachers is that fairness can be interpreted in different ways, and there are often circumstances where none of the options available are ideal. At present, guidance about fairness in classroom assessment is limited. The term is not explicitly defined in existing standards or principles for classroom assessment,[1] and research on fairness tends to relate to standardized or large-scale testing.

Classroom assessment differs from standardized and large-scale testing in several ways. First, it is a dynamic process that relies heavily on teachers’ professional judgment. Second, it draws on spontaneous interactions between teachers and learners (e.g. questioning, feedback) as well as planned events (e.g. quizzes, final projects) for information about learning. The information gathered through classroom assessment serves two main purposes. It has traditionally been used for grading and reporting, which is known as assessment of learning. The use of assessment for learning, where assessment informs teaching and supports learning, is increasingly used in Canadian classrooms.

Fair assessment is complex. With the diversity of learners in Canadian classrooms, the multiple purposes of educational assessment, and the variety of circumstances that arise in assessing learning, there are no one-size-fits-all answers for fairness. However, there are six key areas to consider for fairer classroom assessment:

• transparency;
• opportunity to learn;
• opportunity to demonstrate learning;
• equitable treatment;
• reflective interaction;
• constructive environment.

To provide practical guidance for each of these areas, the following discussion draws on measurement theory, principles for classroom assessment, and empirical research with teachers and learners. [2]

1. Transparency

Learners should know how they are being assessed. Expectations for learning, assessment procedures, and evaluative criteria should be explicitly stated, and assessment results should be clearly explained in order to encourage further learning. These ideas are widely accepted in the theory and practice of educational assessment, and teachers generally agree that assessment should be transparent.[3]

Despite this lack of controversy, there remain three threats to fairness relating to transparency. The first occurs when teachers (and learners in self and peer assessment) don’t recognize or acknowledge all the criteria used in their judgment processes. For example, effort and attendance are often considered in addition to the stated criteria. Student characteristics, including gender, race, cultural background, learning styles, and educational labels, also influence teachers’ judgments. Criteria are more likely to be applied unevenly when hidden, and learners who are less aware of unspoken norms or classroom idiosyncrasies can be disadvantaged. The second threat springs from variance in the amount and quality of assessment-related communication. Some teachers distribute rubrics and assume they are understood, whereas others have frequent conversations with learners about learning expectations and assessment criteria. This communication helps learners understand the task at hand, and shifts attention from grading to learning. Hidden criteria and inadequate assessment talk threaten fairness by obscuring the basis and process of assessment.

In contrast, a third threat occurs with extreme clarity. Criteria may be specified to a restrictive degree that doesn’t allow atypical or unanticipated learning to be recognized or valued. Explicitness should not constrain the opportunity to learn or demonstrate learning. This is a particular concern in classrooms where teaching styles or curricula are not responsive to learner diversity. To avoid these issues, experienced teachers recommend revising narrow criteria, sharing exemplars or generating criteria with learners, and attending to transparency throughout the teaching, learning and assessment process.

2. Opportunity to learn

Opportunity to learn is a complex concept influenced by a wide range of interacting factors, stemming not only from classrooms, schools and educational systems, but also from the broader socio-cultural, economic, and political environment.[4] In research, opportunity to learn is discussed in terms of the past, present, or future lives of learners. Fairness issues are associated with all three perspectives.

When differences in learners’ past experiences are recognized, opportunity to learn is a social justice issue that extends beyond classrooms. Teachers’ beliefs about learners’ past opportunities can result in dramatically different actions. For example, one teacher might respond to learners entering secondary school from a poorly resourced elementary school by providing enrichment opportunities, whereas another might limit activities based on assumptions about learner readiness.

When present practices are discussed, questions about opportunity to learn usually relate to the content of an assessment. Events that reduce the amount or quality of teaching (e.g. inclement weather) cause misalignment between a planned assessment and the learning opportunities that are actually provided. This problem can occur in administering school or district-wide exams. In addition, some teachers suggest that assessment of learning is fairer when it provides a meaningful opportunity to learn in itself, either during the task or through subsequent feedback.

When learners’ future opportunities are considered, concern focuses on the consequences of teachers’ assessment decisions. Status among peers, program placements, access to scholarships, and employment opportunities can all be affected. The pressure to consider extraneous factors in assessment decisions mounts with the stakes involved. Knowledge of scholarship cut-off scores and post-secondary options, for example, can influence decisions about senior students’ final grades. While adjusting certain students’ grades may be done with good intent, it is unfair in a process that uses comparison as the basis for distributing future opportunities.

3. Opportunity to demonstrate learning

Learners should have multiple opportunities to demonstrate learning, so that educators have sufficient information to ensure accurate assessment of learning, especially for high-stakes decisions. Learners should also have varied opportunities to demonstrate learning. This is based on the understanding that learners are diverse and learning is not a uniform process. Pan-Canadian survey results suggest that while many students are given multiple opportunities to demonstrate learning, variety is often limited to a few conventional types of assessment.[5]

In practice, providing multiple and varied opportunities to demonstrate learning is more difficult than it may seem. Two main issues must be considered for fairness. The first relates to the types of assessment methods used. Not only are different kinds of learning captured with different methods, learners’ characteristics influence how they respond to these methods. For example, a quiet student might display a more sophisticated understanding of a novel in a response journal than in a literature circle. Using accurate information about learners and the type of learning involved while planning opportunities leads to fairer assessments. The second issue relates to teachers’ beliefs about learners and learning. When teachers hold low expectations for some learners, or when learning activities lack substance, they fail to provide meaningful opportunities for learning to be demonstrated. This affects the quality of subsequent learning opportunities, and perpetuates a cycle of underserving learners.

4. Equitable treatment

Equality and equity are often confused in educational assessment. The principle of equality, or treating everyone in the same way, underlies standardized testing. Making the content, conditions and scoring criteria the same for all test-takers allows individual or group results to be compared. In contrast, equity involves treating individuals appropriately according to their rights and learning needs.[6]

In classroom assessment, equality requires consistency, whereas equity is associated with responsiveness and differentiation. While consistency and differentiation may seem antithetical, there is a need for both in multi-purposed classroom assessment. Consistency is important in assessment of learning for the same reason that standardization is important in large-scale testing. It would be necessary, for example, if assessment results were used to rank students for awards. The assessment content, conditions and criteria should be consistent for comparisons to be fair. However, equality is sometimes overvalued by both teachers and students.[7] There are often circumstances where strict adherence to the principle is unfair, such as when learners need adaptive technology or other accommodations to fully demonstrate their learning. Furthermore, equal treatment can be counterproductive for some assessment purposes. With a formative or metacognitive function, it is more effective to take individual learning differences and progress into account. Differentiation in assessment for learning is fairer because it allows assessment to serve learners. When classroom assessment is used for multiple purposes simultaneously, balancing consistency and responsiveness can be challenging. The greatest threat in this process is when equitable assessment is not a conscious or explicit goal, and decisions are influenced by stereotypes or personal values. These can be so entrenched that they are overlooked even when the intent is to be fair.

5. Reflective interaction

Fairness in classroom assessment involves more than following protocol – it requires thought. Guidance for classroom assessment contains innumerable recommendations for reflection by teachers, and two topics in particular stand out for fairness. First, reflection about biases and values is critical. Fair classroom assessment depends on professional judgment, which is informed not only by knowledge and experience, but also by moral beliefs and cultural norms.[8] Without reflection, biases can creep into teacher-made tests and undermine classroom interactions. This may occur, for example, when Aboriginal stereotypes are accepted and local culture is not valued in schools. Learners should also be encouraged to think about the basis of their judgments and the impact of their comments when they engage in peer assessment.

A second topic for reflection is the purpose of classroom assessment.[9] Purpose is particularly salient for fairness because it dictates the decision-making framework. Assessment of learning should be criterion-referenced, which means that learning is compared to specified learning expectations, usually to determine grades and write report cards. Assessment for learning may be criterion-referenced and learner-referenced. It takes learning targets, individual needs and progress into account in the ongoing process of teaching and learning. Fairness is threatened when the assessment purpose and framework are mismatched. For instance, boosting a final grade for a hard-working learner is not fair because it conveys an inaccurate message about achievement, which can have a negative impact in the long term. Another problem occurs with an informal type of norm-referencing. When teachers compare learners to each other (rather than comparing learning to curricular expectations), their framework shifts with the range of ability in a class or their memory of previous learners. Teachers can avoid these issues by thinking about the purpose of an assessment and the biases or values at play in the process.

6. Constructive environment

Interest in the relationship between assessment practices and the learning environment has increased with the recognition of the social nature of classroom learning. Recent research suggests that while assessment practices affect the quality of the learning environment, the reverse is also true. The relationship between assessment for learning and fair assessment may also be mutually supportive, meaning that in an environment where one is encouraged, the other is likely to occur.[10]

The ideal features of a classroom environment that supports learning through fair assessment may depend on multiple factors, such as the age of the learners, the curriculum, and the broader educational context. At present, two essential needs are evident. The first is for teachers to be aware of power dynamics in assessment. The authority of teachers in classroom interactions is well recognized, especially in the assessment of learning where they must elicit, examine and judge learning. Some teachers voluntarily use techniques such as blind marking in response to this authority. The power of learners is not as frequently acknowledged, but learners do influence each other and their teachers. When they are involved in assessment for learning, even more control shifts from teachers to learners. This does not necessarily make assessment fairer. When power dynamics are not considered, some assessment methods that aim to support learning are less fair than traditional tests. For example, the fairness of asking learners to reveal what they know (or do not know) through self-assessment, and then using that information to their detriment, is highly questionable.

A second and related need is for trust and respect. Learners in different contexts express the desire to be treated with respect in assessment interactions, and trust is central to their willingness to engage in learning and assessment.[11] Trust and respect affect and are affected by classroom assessment. Proactively nurturing these qualities within the classroom allows teachers to develop an environment in which constructive feedback can be given and received by learners.

Fair classroom assessment is a professional responsibility that should not be a matter of chance. By attending to these six key areas in their planning and classroom practices, teachers can make assessment fairer for every student.

Six Steps to Fairness

Engage learners in conversations that clarify all expectations and criteria used while assessing learning, and focus assessment-related talk on learning rather than on performance for grades.

Plan or design assessments so that they are preceded by ample opportunity to learn, and adjust an assessment if learning opportunities are disrupted by unforeseen events. Encourage learning before, during, and after assessments.

Provide multiple and varied opportunities for diverse learners to demonstrate learning. Choose or design tasks that are meaningful for the particular learners and suitable for the type of learning involved.

Aim for equitable assessment. Balance between being consistent and being responsive to individual needs by taking the intended purpose of an assessment and probable use of the results into account. 

Reflect about the purpose of assessment interactions and any values or biases at play. Try thinking backwards from a judgment (what information about the learner or learning did I use and why?). It may be helpful to discuss your assessment practices with a supportive colleague.

Nurture a constructive learning environment through modelling and explicit teaching. Encourage fair assessment processes and judgments by all members of the classroom.

First published in Education Canada, June 2013

EN BREF – Tous – enseignants, dirigeants, parents et élèves – veulent que l’évaluation scolaire soit équitable. Or, une évaluation équitable est une question complexe qui peut poser des défis aux enseignants. En raison de la diversité des apprenants dans les classes canadiennes, des multiples objectifs de l’évaluation en éducation et de la multiplicité des circonstances entourant l’évaluation des apprentissages, il n’existe pas de solution unique pour assurer l’équité. Les enseignants peuvent rendre leurs évaluations plus équitables pour les élèves en suivant six recommandations : a) une communication transparente; b) une planification et des ajustements rehaussant les possibilités d’apprentissage; c) la possibilité pour les élèves de démontrer leurs apprentissages de façons multiples et variées; d) un traitement équitable plutôt qu’égal; e) une réflexion à propos de l’objectif des interactions en évaluation, des valeurs ou des préjugés; f) un environnement d’apprentissage constructif pour tous les apprenants.

[1] Joint Advisory Committee, Centre for Research in Applied Measurement and Evaluation, Principles for Fair Student Assessment Practices for Education in Canada (University of Alberta, 1993), www2.education.ualberta.ca/educ/psych/crame/…/eng_prin.pdf; Joint Committee on Standards for Educational Evaluation, The Student Evaluation Standards: How to improve evaluations of students (Thousand Oaks, CA: Corwin Press, 2003).

[2] For a detailed review of this literature see R. D. Tierney, “Fairness in Classroom Assessment” in the SAGE Handbook of Research on Classroom Assessment, ed. J. H. McMillan (Thousand Oaks, CA: SAGE Publications, 2013), 125-144.

[3] For example, in the Netherlands and the U.S.: K. Ploegh, H. Tillema and M. Segers, “In Search of Quality Criteria in Peer Assessment Practices,” Studies in Educational Evaluation 35 (2009): 102-109; S. K. Green, R. L. Johnson, D. H. Kim and N. S. Pope, “Ethics in Classroom Assessment Practices: Issues and attitudes,” Teaching and Teacher Education 23 (2007): 999-1011.

[4] P. A. Moss, D. C. Pullin, J. P. Gee, E. H. Haertel and L. J. Young, Assessment, Equity and Opportunity to Learn (Cambridge, UK: Cambridge University Press, 2008), 338-340.

[5] Council of Ministers of Education, Canada, “PCAP 2010: Contextual report on student achievement in mathematics” (Toronto, ON: 2012), 158-163; D. Hunter, C. Mayenga and T. Gambell, “Classroom Assessment Tools and Uses: Canadian English teachers’ practices for writing,” Assessing Writing 11 (2006): 42–65.

[6] S. Messick, “Consequences of Test Interpretation and Use: The fusion of validity and values in psychological assessment” in Problems and Solutions in Human Assessment, ed. R. D. Goffin and E. Helmes (Norwell, MA: Kluwer Academic Publishers, 2000), 3-20.

[7] Strong beliefs about equality in the classroom have repeatedly been noted in empirical research with students and teachers. Two examples are C. M. Brighton, “The Effects of Middle School Teachers’ Beliefs on Classroom Practices,” Journal for the Education of the Gifted 27 (2003): 177-206; C. Dalbert, U. Schneidewind and A. Saalbach, “Justice Judgments Concerning Grading in School,” Contemporary Educational Psychology 32 (2007): 420-433.

[8] G. Biesta, “Values and Ideals in Teachers’ Professional Judgement” in Changing Teacher Professionalism: International trends, challenges and ways, ed. S. Gewirtz, P. Mahony, I. Hextall and A. Cribb (Abingdon, UK: Routledge, 2009), 184-192.

[9] L. Earl and S. Katz provide considerable guidance in Rethinking Classroom Assessment with Purpose in Mind: Assessment for learning, assessment as learning, assessment of learning (Western and Northern Canadian Protocol for Collaboration in Education: Governments of Alberta, British Columbia, Manitoba, Northwest Territories, Nunavut, Saskatchewan, and Yukon Territory, 2006). http://www.wncp.ca/english/subjectarea/classassessment.aspx