Tag Archives: grade descriptors

The tantalisation of standardisation: It’s Teenage Mutant Ninja Turtles all over again… and again… and again…

Back in the early 90s, my older brother managed to get me a second-hand Nintendo Entertainment System (NES). It came with two games which, as I had no money to buy any others, occupied much of my time for months on end. One was the classic arcade game Kung Fu Master and the other was Teenage Mutant Ninja Turtles.

Whilst Kung Fu Master didn’t take long to, er, master, TMNT was a different beast. It was impossibly difficult. It was so difficult that, along with a handful of other NES games, it contributed to the phrase ‘Nintendo hard’ entering the English language. Some of the levels were almost unplayable (I seriously think the one with the van was solely created just to crush the spirit of children), but what made the whole experience impossible was the fact that this was in an era when there was no ‘save game’ feature on consoles. So every time I lost the game, I had to start again at the beginning. I don’t want to work out the number of hours I threw away making barely perceptible progress on TMNT.

But as the saying goes, ‘When I became a man, I put away childish things’. Whilst anyone who even vaguely knows me would know that this obviously isn’t even slightly true of me, I have definitely moved on from wasting endless hours trying to overcome such frivolously difficult tasks – tasks where I ultimately get nowhere and have to start right back at the beginning again after each attempt. That is until I became an English teacher. Because, since I became an English teacher, I’ve had to take part in standardisation. Regularly.

Standardisation, to the uninitiated, is the act of moderating assessment with colleagues in order to establish a standardised level of accuracy in grading. Seems like a wholly appropriate thing for any English department to do, particularly in the days of coursework and controlled assessment. The problem is that standardisation, whilst well-intentioned and seemingly necessary, is a bit like Teenage Mutant Ninja Turtles on the NES.

Because moderating and standardising assessment, certainly in English, doesn’t mean we get standardised grades. Like TMNT, we seem to make progress whilst we are standardising: agreeing on grades and reaching some kind of harmony with our marking. But also like TMNT, the next time we come back to the marking, we have to start all over again: much of what we gained in the standardisation process is lost.

If you don’t believe me, take a look at the Ofqual report, ‘Marking consistency metrics’, on the quality of marking in general qualifications. Bear in mind that examiners of GCSEs and A Levels undertake more rigorous standardisation than your regular classroom teacher, the findings of the report are pretty depressing. For their report, Ofqual put seeded papers (those that have been pre-marked and assigned definitive marks) out to be remarked by the team of employed examiners. Below is a table showing the probability of a candidate being award the definitive mark. For English Literature, it’s around 50%. History isn’t much better at around the 60% mark.

From the Ofqual report: ‘Boxplot of the probability of a candidate being awarded the definitive grade for a range of units. The mean probability for each subject is denoted by the white triangle.’

The elephant (or turtle) in the room when we standardise is that, when left to our own devices, much of what we gained in standardisation is lost – lost to unconscious bias, lost to the subjective nature of grade descriptors, lost to tiredness, lost to caprice, lost to the fact we might subconsciously compare against the previous piece of work we marked.

And yet we still seem to give up lots of time to standardisation.

The idea that, by practising assessing, and by moderating with colleagues, we are standardising our marking and getting more accurate is a tantalising one. And tantalising is the perfect word for the whole process, as its very etymology brings to mind another good analogy for the largely futile activity. For the word derives from the character in Greek myth, Tantalus, who was punished by his father, Zeus, in a rather spectacular way. Tantalus, a mortal, was invited to dine with the Gods on Mount Olympus. He wanted to test whether the Gods really did know everything, so (obviously) he decided to kill his own son, Pelops, chop him up, cook him and serve him up to see if they knew what they were eating. The Olympians immediately knew what had happened (except Demeter, who was probably looking at phone and so wasn’t paying full attention). Zeus then dished out the most delicious punishment: Tantalus was made to spend eternity in a pool of water which sat beneath trees hanging with bounteous fruits just above his head. But every time he bent to drink the water, it would drain away so he couldn’t get to it, and every time he tried to reach the fruits above his head, they would rise up away from his grasp. Hence: tantalising – ‘tormenting or teasing with the sight or promise of something unobtainable’.

That’s the perfect analogy for standardisation: it torments and teases us with a promise of accuracy, something that is ultimately unobtainable. We should probably be cautious about investing too much time on it. Which is exactly what my mum kept telling me about Teenage Mutant Ninja Turtles. Mums are always right.

(Yes, gamers, I know it was called ‘Teenage Mutant *Hero* Turtles in the UK; the original US title is used here to avoid quizzical responses from non-European readers.)

Advertisements

A Glossary of U.K. Education (Vol. 3)

Following on from volumes one and two, here’s the latest edition of our education glossary.

Bennett, Tom

/bɛnɪt, tɒm/

noun

Scottish outlaw and folk hero; known colloquially as ‘Tam’ Bennett, he formed the researchED clan and led the VAKobite Rebellion against the neuromyths laying claim to the throne of pedagogy; he ultimately overthrew the House of Brain-Gym and restored evidence to its rightful place.

Bloom’s taxonomy

/bluːmz takˈsɒnəmi/

noun

a hierarchical model of classification which organises learning into six levels of complexity: remember, understand, apply, analyse, evaluate, create; once you have overcome all of these levels, you must ultimately defeat the end-of-level boss: obfuscate.

Blue’s taxonomy

/bluːz takˈsɒnəmi/

noun

a hierarchical model of classification which organises items on a scale of least irritating to most irritating, based on the members of the boy band Blue: Simon Webbe (least irritating), Duncan James, Antony Costa, Lee Ryan (most irritating); e.g., “The consultant delivering that CPD was absolutely Lee Ryan.”

caffeine

/ˈkafiːn/

noun

life-force of teachers; they can cut our budgets, they can freeze our pay, but if they come for our coffee and tea, they’ll have to prise it from our cold, dead hands.

Christodoulou, Daisy

/krɪˈstɒduːluː, ˈdeɪzi/

noun

the only prominent educationalist who is most commonly referred to by their first name alone.

fair funding formula

/fɛː ˈfʌndɪŋ ˈfɔːmjʊlə/

noun

unfair funding formula.

grade descriptors

/ɡreɪd dɪˈskrɪptəz/

noun

occult apparatus used for supernatural divination; a form of cleromancy in which prophets will look over a document and then interpret it using the grade descriptors to guide them to a grade, which will then be challenged by another prophet who used the same descriptors to come up with an entirely different grade; a process of debate will follow until the prophets can agree on an interpretation of the descriptors that angers the spirits the least.

Hirsch, E.D.

/hɛːʃ, ˈiː ˈd/

noun

educationalist and academic; be honest, you think his name is Ed, don’t you? I mean, maybe not consciously, but subconsciously, you sort of think of him as Ed Hirsch, don’t you? Yeh, you do.

interactive whiteboard (IWB)

/ɪntərˈaktɪv ˈwʌɪtbɔːd/

noun

a large interactive

display that,

when written on with an

interactive whiteboard pen,

displays the writing wherever the                                               hell

it

wants.

Even after calib

ration.

Mantle of the Expert

/ˈmant(ə)l ɒv ðə ˈɛkspəːt/

noun

educational approach in which novices spend their time pretending to be experts so that they can remain novices for longer.

Slough of Despond

/ˈsl əv dˈspɒnd/

noun

the filthiest, most festering, fungus-ridden mug in the staffroom, as mentioned in John Bunyan’s The Pilgrim’s Progress: “This miry Slough is such a place as cannot be mended; it is the descent whither the scum and filth […] doth continually run, and therefore is it called the Slough of Despond.”; it is traditionally discarded in the waste bin at the end of every term, only to magically return in another form within a few weeks of the following term.

snow

/snəʊ/

noun

the gift given to tired teachers from the benevolent gods every few years as a reward for their hard work in educating the children (cf. wind, which is a punishment handed down to teachers from the vengeful gods for not keeping up with the marking).

Summer Holiday

/ˈsʌmə ˈhɒlɪdeɪ/

noun

1963 British film starring Cliff Richard, in which he plays a character who wanders about feeling utterly purposeless at the start, then writes himself a to do list of household jobs he’s been putting off all year; he goes on to read a couple of books, binge watch some boxsets, fall asleep in the afternoon a bit, and he finally goes into work to put some displays up on the walls; just as the credits roll he suddenly realises that he hasn’t done any of the household jobs on his to do list.

Teach First

/tiːtʃ fəːst/

noun

charity which focuses on giving bankers hearts.

wind

/wɪnd/

noun

powerful chemical catalyst; just one part wind mixed with 100+ parts children will cause uncontrollable agitation and ebullition of said children.

wine

/wʌɪn/

noun

a form of neuralyzer (the memory-wiping device made famous by the Men in Black film franchise); it is used by teachers on themselves each night in order to forget the ignominy and upset of being told to “!@$# off” or that “your lessons are boring”; sometimes these things are even said to them by pupils.

I ❤ January 2015

When TV shows run out of ideas, they fall back on that old faithful: the clip show compiling all of the ‘best bits’.

And when they run out of their own TV shows, there are always clip shows made up of the best bits of other people’s shows: the I ❤ 1984 (etc.) model, featuring talking heads from D-list celebrities reminiscing about the time that dog said “sausages” on That’s Life.

As a D-list blogger myself (what do you mean I’m getting above my station?), this regurgitation of other people’s brilliance is the perfect model for me to reminisce on the best blog posts of each month (with the added implication that I’ve run out of ideas).

(In all seriousness, I got to the end of 2014 and realised I’d read so many great blogs but not really collected them anywhere. So this monthly blog is a way for me to compile an anthology of some of the best reading in one place and be able to access it when I want to call upon it again.)

So without further ado, this is my ‘clip show’ of the blogposts that I read and enjoyed the most in January…

  • The nonsense of the grade descriptors by @chrishildrew: Chris went down the rabbit hole of grade descriptors and has exposed us to the mad tea party. As Alice said, “How puzzling all these changes are! I’m never sure what I’m going to be, from one minute to another.”
  • Why I Hate Highlighters! by @HuntingEnglish: I like this because Alex confirms what I think I might have always feared, but never quite confronted: highlighters often put a garish neon gloss over a lack of actual learning. Rumours are unconfirmed that this is the first in a series of ‘Why I Hate…’ blogs, which will feature other such objects of Alex’s anathema as children and Maths teachers. (For balance, and because I like him, this is highlighter advocate @jon_brunskill‘s rebuttal.)
  •  Some Problems With “Action Research” by @Bio_Joe: Thanks to this brilliant post by Joe, I’ve now added the word significant to my list of words-that-are-used-in-a-way-which-often-leaves-their-actual-meaning-behind-in-order-to-promote-a-pedagogy (see impact, evidence, research, etc.) The “study” Joe picks apart here comes from a website riddled with spurious arguments and “research” in the name of “evidence”. Which is a shame because it is an area I’d like to see some reasoned thought around.
  • Can we teach students to make inferences? by @atharby: Andy precisely and eloquently pinpoints the very reasons why teaching thinking skills is largely unhelpful, and why building student knowledge is a much more effective approach. I wish I’d had this to hand when I sat through a cognitive acceleration training course that promoted thinking skills in English recently.
  • How do we get them reading? by @katie_s_ashford: Katie generously shares the fruits of her scrutiny on the research and approaches to solving “the problem of reading”. These systematic and practical ideas are absolute gold – send this to your literacy coordinators/English department/SLT/everyone now.
  • Undermining teachers is easy by @LearningSpy: The blogdaddy David Didau reiterates the necessity for schools to master behaviour as requisite for learning, and decries the damaging line of thought (avowed in this instance by a school inspector, no less) that states that good behaviour is merely a product of good teaching.
  • A lesson is the wrong unit of time by @BodilUK: A second blog from Bodil, in which she questions why our discourse and measurement always revolves around ‘the lesson’ as a unit, when the reality of learning expands way beyond that unit’s boundaries. She’s absolutely right, as usual.
  • I Did Not Speak Out by @SurrealAnarchy: Martin’s writing always provokes deep thought, and this clever channelling of Pastor Niemöller is a stirring illustration of the constantly shifting focuses and measurements in schools (and the impact of these on pupils and teachers).