Why Making Sense Of Mock Results Isn’t Easy

Mock Exams

This article first appeared in HWRK Magazine in November 2021. You can read the article on the HWRK Magazine website here.

“Correlation is not causation”.

I’ve been thinking of using this as a mantra. Far too often, even those of us who are aware of its truth can fall prey to this logical trap. After all, how many times have you tried out a new teaching technique, with students seemingly succeeding as a result, only to find out later on that they only really made progress because of your newfound enthusiasm, rather than the clever new strategy you used?

The key question is, how do we separate the truth from the noise? We need to ask the same question about mock exam results.

As we move into “mock season”, we should be especially mindful of the correlation/causation problem. The whole point of Autumn mocks is to find out what students do or don’t know, what they can or can’t do and to work out what I need to do next, in response to their performance.

Making Valid Inferences is the name of this game and it’s a lot harder than it seems.

Just imagine this: a student answers a “describe” question with a one-line response, when they should be writing a detailed paragraph or two. What should we make of that as their teacher?

  • Have they misunderstood the amount of detail required?
  • Do they have gaps in knowledge?
  • Is their understanding accurate, but shallow?
  • Did they merely guess the answer correctly without actually knowing it?
  • Is their response just a regurgitation from a revision sheet?

Without an accurate answer to these questions (and more besides), our next move may not have any impact. So what should we do?

Well, let’s look at the responses from the whole class:

  • Do many other students struggle similarly on that question?
  • Are other students having issues with that same exam “skill”, e.g. do they evaluate instead of describe?
  • Did they all run out of time?

Or is it actually more complex than that?

Sometimes there’s a mixed response from different students across the class. Do students from one group perform better or worse than others? A change in seating arrangements might help. But then again, it might not.

Maybe it was the weather that day. Did a wasp fly into the room during your explanation? Were your students in a bit of a rush after being late from PE? Was there a funny smell from the farmer’s field next door that students kept getting distracted by? Did you (without a hint of irony) forget to set a recall homework task on the topic where they underperformed?

In other words, did the problem occur during the teaching, rather than during the exam?

A Question Level Analysis can be helpful, but it won’t always provide the answers that we as teachers need. A good QLA can still only give you a limited amount of information. The information you actually need often comes from your memory of teaching that topic at the time.

What was it that helped or hindered your teaching? This might be a resource issue, a timetabling one, a staffing conundrum, or something on a whole school level, largely beyond the control of the class teacher or Head of Department.

It might even be that your own knowledge just wasn’t strong enough on that topic. That’s an uncomfortable thought, isn’t it? Well, it shouldn’t be. And we can address it without stigma, shame or professional embarrassment. In fact, I’d argue that if we are teaching a challenging curriculum, then from time to time we should fully expect it and actually embrace it in our practice, both individually and as a department. Think back to when you taught that topic: how strong was your subject-knowledge? And are you the best judge of that?

To tackle post-mock issues then should be a collaborative effort, not siloed off for a Head of Department or a Key Stage Coordinator to deal with alone. As a departmental team, it is worth discussing not just “how well did they answer question 8”? but also “how well did we teach the students to be able to answer question 8?” By posing the question in this way, we are much less likely to make assumptions about the student’s answer and much more likely to find the true reason for their response. We should discuss and model our own in-class explanations, how we scaffold and how we assess as we teach, checking for misconceptions and encouraging detail and nuance in students’ responses.

Having these discussions also stops us from letting ourselves off the hook. Much of a student’s attainment is down to things that occur beyond the walls of our classrooms and this is why holding teachers solely accountable for exam results is highly problematic. But we are responsible for how we teach and this impacts student responses in exams in arguably the most significant way. If we have taught it well, the students will typically perform well in assessments.

There are schools across the country, whose cohorts are classed as “disadvantaged” in various ways, but who also routinely outperform other schools whose students “have it easier” (at least on paper). This comes down to the teaching.

As Dylan Wiliam puts it, “Every teacher needs to improve, not because they are not good enough, but because they can be even better.”

So, scrap “correlation is not causation”. My mantra should simply be “Keep improving my teaching”. Everything else is just noise.

You can read more posts like this in HWRK Magazine | The Essential Magazine For Teachers

On Routines, Running, Greasing the Wheels and (Crucially) Biscuits

man running on side of road

This article first appeared in the September 2021 Edition of HWRK Magazine.

I may not look like it, but I love running. When I’m at my “peak” fitness level (which is not only rare, but also far less impressive than it sounds) I’m able to get myself up at 5:30am, throw on my running gear and do a reasonable 5k. This lasts about two or three weeks, then I run out of steam. I make excuses to miss the odd day, which quickly spirals into a string of missed days. Right now, my non-running streak is at the several month mark. I desperately want to run though, so why on earth can’t I stick to my routine?

Well, I do have a theory. I can’t prove it, but whenever I tell someone, it seems to resonate with them. It’s got something to do with biscuits. 

My running “routine” (if you can call it that) involves a lot of thinking about not running, then talking myself into running, then forcing myself to actually do it. If there are any obstacles in my way, like having to do something else urgently (or even not urgently), or if I’ve had a heavy meal beforehand, or if I’m tired, then I justify to myself that I don’t need to do it this time. I’ll just do it tomorrow and everything will be fine.

But, it’s not fine and it gets much worse. My routine is easily destroyed if I do something simple, like eating a biscuit. This might seem like a small thing to you, but what eating a lovely little biscuit does is signal to me that “you know what? Running isn’t that important. You can just eat biscuits. Nobody will mind”. It’s a seductive voice. Think M&S Christmas food advert voice. In the end, the biscuit always wins and I’ve found the perfect excuse to stop running. If I’m eating biscuits, then I’m already not being healthy, so I might as well quit exercising too. Game Over.

What’s this got to do with teaching though? Well, in September, we all like to start with a clean slate and embed good habits and routines, both for ourselves and our students. These take a little getting used to, but with a bit of effort, they stick and after a couple of weeks, you might even think “I’ve nailed it”. Cue, the biscuits.

Once you think you’ve embedded your routines, like setting frequent retrieval tasks, enforcing a behaviour management strategy, or keeping on top of your emails, you will inevitably hit a point where you think, “I’ll just not do it this time. I’m… too busy. Plus, I’ve already stopped doing that other Very Important Thing that I was supposed to be doing. I’m only human and sometimes I need a break… That’s it, I deserve a break” But at this point in the year, it’s a trap. It’s the M&S Christmas food advert voice again. And it really isn’t on your side. 

Once you miss a day or two, or a week, or longer, your routine will naturally fall by the wayside. But worse, it will then be harder to resurrect it than it was to start it from scratch. It’s tainted by “failure” now. You’ve lost your streak. It doesn’t have that special shiny new object feel that it had, back in that first week in September, that felt so motivating.

Fear not though, there’s something I’ve found helpful in exactly this situation: Plan for the inevitable breakdown of your routine. 

Make a decision, ahead of time, about what steps you will take when your routine goes to pot. Think about how you’ll feel when it happens and also what your lack of routine will do to your day. Then, make a To Do List which deals with those issues. Keep it brief, actionable and realistic. Then have it ready for when the inevitable happens. You’ll thank yourself.

Tonight, before I wrote this, I enacted my own To Do List: I laid out my t-shirt, shorts, socks and trainers. I’ve set my alarm for 5:30am and, in a rare stroke of genius, I even set the timer for the coffee machine to come on. I am going to run. There’s nothing to get in my way. I’ve removed the metaphorical biscuits.

Then, tomorrow night, I’ll just lay out my gear again. It’s much easier to get back into my running routine (and stick to it) once I’ve greased the wheels a little by taking the effort out of the period where I’ll be at my most vulnerable to giving up, i.e. bleary-eyed at 5:30am. 

So, grease your wheels too. Automate and schedule your recall tasks in Google Classroom, Microsoft Teams, or whatever platform you use. Have a handy phrase that sums up the behaviour principle you need to keep remembering and repeating. Display it on your classroom wall so neither you nor the students can miss it. Set aside a time of day where you are undisturbed and ONLY allowed to deal with emails. Remove those biscuits.

But most importantly, when it goes wrong, just remember: If I can get back into running (and hopefully one day soon my favourite work trousers), then you can get back into your good routines. After all, you chose them for their usefulness in reducing your workload, or because they help students make more progress. They may even serve an important moral purpose. To thrive in this job, you need your good routines to stick and you really can do it. 

Best of luck (and don’t forget about the biscuits).

Read the latest edition of HWRK Magazine here!

%d bloggers like this: