You Can’t Fatten a Hog by Weighing It
Grading for quality in writing
How can we define and grade quality in writing? Ken Draayer has wrestled with this question for 30 years, and concludes that it cannot be captured by a rubric or a list of features. He demonstrates how checking off a series of requirements, though it might earn you 100% on the marking schema, does not add up to good prose.
From accountability reforms beginning in the 1990s in Ontario, I learned that grading is the measurement of student, school and system results to strengthen educational management. From Robert Pirsig’s Zen and the Art of Motorcycle Maintenance I learned that grading is a search for quality and how to care for it.
I had not read Pirsig when I began grading English papers at a private school in the 1970s and had to define “quality” in student work. My undergrad university training and subsequent experience in journalism suggested it was some combination of subject knowledge, coherent organization and audience appeal. On the evening of my first grading experience, however, these categories proved unhelpful as they collided with actual student writing.
I decided to grade by approximating quality-as-a-whole using the stairs leading up to the bedrooms in the house – the further up the stairs, the higher the quality. The top stair, I decided, represented 90 percent. The steps below declined by 10 percent each until I reached a personal floor for failure at 40 percent – my compassion at the bottom level countered by severity at the top. I had never scored 100 percent on an essay; why should my students?
My approximate staircase kept me on task to completion. I then introduced the common categories – content, structure, language – and fudged those more precise 68s, 76s and 83s until I was ready to hand the papers back – though perhaps not so much to respond to I don’t understand this mark (and neither will my parents), and I’ve never got less than 80 in English. Then and after, the experience of grading remained unsettling, always hedged round by doubts.
Zen and the art of grading
In Pirsig’s novel, grading is an ethical dilemma. Phaedrus, a teacher of rhetoric, strives for a shared understanding of quality with his students and confesses his doubts about grades and their actual relationship to quality. At one point, he drops grades entirely. Rules about content, structure and language, he tells his students, were imposed on writing after it was done. Teachers who prescribed, and students who wrote by prescriptions, produced writing that “had a certain syrup, as Gertrude Stein once said, but didn’t pour.” Quality was the goal, but “when you try to say what quality is, apart from the things that have it, it all goes poof.”
In my 30 years teaching in high schools and in pre-service courses at Brock University’s Faculty of Education, I would sometimes read six assignments to set my “spidey sense” about quality before marking. Other times I would read and try to maintain consistency and focus by commenting on quality, one category at a time. Like Pirsig, I wrestled with the supposed relationship between quality and my grades.
Grades and perfectibility
And what about grades in math and the sciences, where 100 percent on a test or assignment is entirely possible, and a final of 98.5 not unheard of? In the humanities, we seem to be stuck with imperfection. Why does my 72 percent always require turgid explanations, and elicit wrinkled brows and “whatever”? Why should English marks be such a dead weight on averages needed for university or college applications?
I decided to thwart this notion that quality, in English, was so infected by innumerable and inscrutable sins that perfect marks were out of the question. Writing for Indirections – a journal for English teachers in Ontario – I described my new “publishable” grading method, introducing the prospect of 100-percent essay results to my students.
I commissioned from a local trophy shop a rubber “publishable” stamp and inkpad and explained to my students that, while I would return papers with estimated marks, I would read them wearing an Editor’s visor, looking for that “something,” that quality that attracts readers to read and editors to say, “Let’s publish this!”
The “publishable” deal was this: If you agreed to one or two editorial conferences with me and – in the interests of understanding quality – to apply revisions intended to make the piece more clear and readable, I would then guarantee a final mark of 100 percent to reflect our mutual striving for quality. Real writers don’t work alone. They have editors. It’s a team thing.
Changing the teaching relationship
The first recipient of the “publishable” stamp was a piece on hacking. The writer was keen and knowledgeable about his theme, but not so much about his writing. But my “spidey sense” again said quality was there and, confident of its eventual improvement, I gave it a 40 percent and exchanged my identity as marker for editor which, if you think about it, is a significant shift in the politics of the classroom. The final product, though not a silk purse, got 100 percent. And I got my first accomplishment as editor.
Perhaps this sounds like a disingenuous game (mark inflation!). But it did produce interest and thoughtfulness about how quality comes about in writing. In some iterations I added that students could request the publishable stamp. I created student editor/writer partnerships and suggested they should share the resulting grade, both being responsible for final results. I recall adding once, for self-protection, that I would only stamp six assignments “publishable” in any one batch because of the increased time imposed on me and on the student writers. I’d like to say that the method had such appeal that I was inundated with editorial work and giving out 100s by the 100s. But I was not. Human nature saved me. The line-up to do drafts and revisions was short.
These variations on the publishable method changed my writing instruction for good. One final variation I had not entirely foreseen. I had a set of papers I’d been putting off marking. One night I put the stack beside the computer and decided, in my Pirsig-ness about Quality, to set aside my red pen and take up the Editor’s visor for every writer. As I read each paper I typed editorial responses to purpose, to the ideas, to coherence, to language – to any aspect, in fact, where I thought I might coax out more quality. I especially conveyed my enthusiasm for their enthusiasm, or suggested a little more enthusiasm, and I remember thinking, when done, what a delight this had been for me – but I had no marks!
Next day, returning the papers, I struck a bargain. Would it seem fair, I asked my students, given the extensive editorial comments and the opportunity to improve results, that they would agree to give me time management considerations and accept an unexplained mark on the final draft? No categories. Just a quality-as-a-whole mark? And they agreed. After all, what could be more fair?
Grading and accountability
Grading that supposes strict measurement by rule and precept imposes its own game (less worth playing), in which standards and rubrics presume quality and imply that teaching professionals need not search for it with their students. Under the weight of measurement, quality goes poof.
Grading practices reflect the personal knowledge, intelligence, and ethical sense of teachers. Students know this. Uniform standards and rubrics sweep aside the personal discussion and experimentation with this necessary but complex part of teaching and learning. In its place we have Ontario’s current “Achievement Chart,” the mother of all rubrics in which, over four levels, there is simply limited, some, sufficient, or thorough quantities of skill or knowledge. Bad-a-boom, bad-da-bing: quantity, not quality. The Ministry of Education could be forgiven for thinking this now defines teacher practice.
Here’s the kind of grading and teaching modeled by accountability: Early versions of the Grade 10 Test of Literacy contained instructions for an Opinion Piece. Students were given a theme (e.g. the Welland Canal) and a list of related facts. The task: write a three-paragraph response organizing selected facts – a kind of Lego approach to writing. Several facts about the canal were listed. A teacher in our board, experienced in EQAO marking, delivered a workshop revealing the Opinion Piece marking rubric and its use by trained markers in Toronto.
Uniform standards and rubrics sweep aside the personal discussion and experimentation with this necessary but complex part of teaching and learning.
On the basis of this workshop, a local teacher, so armed and busy teaching to the test to raise his school’s results, devised the following strategy to constrain the Toronto markers, using their rubric, to award his students 100 percent on the opinion piece. His instruction to them was:
- In the list of facts, find three good things, OR three bad things about the Welland Canal. Decide on your opinion. Does the Welland Canal “suck” or does it “rock”? Your decision is your main idea – your opinion.
- First paragraph (indent): One sentence: State that opinion. Either the canal sucks or it rocks.
- Second paragraph (indent): Write only three sentences. In each sentence use the word “because” with one of the facts. For example: The Welland Canal rocks because it moves boats from Lake Ontario to Lake Erie.
- Final paragraph (indent): Repeat your opinion in one sentence this way: So, as I said, the Welland Canal sucks (or rocks).
Using grading for accountability and neglecting any notion of quality, thousands of Ontario students were declared “literate” and the system of test and measure was declared a success. But a wise Curriculum Superintendent once reminded me of the old homily: “You can’t fatten a hog by weighing it.”
First published in Education Canada, March 2019