Black Box Thinking for Teachers

Black Box Thinking for Teachers

What is “Black Box Thinking”?

Black Box Thinking is a philosophy which allows learning to emerge from mistakes.

The phrase was coined by Matthew Syed in his excellent book of the same title, where he examines performance and critical self-evaluation in sport, aviation, politics and many other fields. He took the term from the “black box” flight recorders fitted to aircraft, which contain vast amounts of data, to be used to inform future improvements. They are used extensively, but especially following poor performances. These could result from human error, failures of systems and procedures and unexpected events.

How does black box thinking apply in education?

In education, just as in aviation, we continually train ourselves and others, ensuring consistently high performance. But despite the time put into this training, students can still underperform in exams. Schools and inspection bodies collect this data, containing a wealth of information to guide current and future performance. But I’m not certain that we use this information effectively.  After all, which information should we act on and how on earth should we act on it?

When teaching doesn’t work…

A few years ago, Steve, a friend of mine working in another school, called me on A Level Results Day. He was in shock. For the last few years, his students had achieved excellent exam results and he was considered by many to be an “outstanding teacher” (I hate that phrase!). This year, however, a number of his students had “failed”. By “failed”, he meant that they had passed, but had significantly dipped below their expected grades.Steve had to account for this dip in his post-results analysis that he had to present to the Headteacher. But only two months earlier he had predicted much higher grades. How could he have got it so wrong?

In essence, he had assumed that because he had always been right about his students in the past, he was able to draw similar conclusions about his current students. Unfortunately, he was looking at the wrong data or at least interpreting it in the wrong way.

Steve’s current students were not in any way “weaker” than in previous years. Nor had his teaching changed much. But he HAD missed one crucial point. The STUDENTS were different. He had forgotten to take this into account. This caused him to infer that the data he had used effectively last year was just as relevant for this year’s students. Steve was wrong.

When the “data” doesn’t add up…

We are all familiar with the use of assessment results to inform our understanding of how students progress towards their targets. However, those results do not “measure progress“. They are a proxy, something which may indicate progress but which is not synonymous with it. Steve believed his assessment procedures to be rigorous. He used a range of assessment questions from the exam board’s past papers. He was a seasoned examiner and was a competent judge of student responses. But he was ignoring something crucial. Steve focused entirely on improving the skills and techniques used in answers to exam questions. It made no difference in the end.

Steve recalled some of the papers from the exam board to see what had gone wrong. He assumed that the students had ignored the techniques he had taught them. How could they have forgotten the special mnemonics they had constructed together? Had they not written using PEE paragraphs? Did they follow up each of their ideas with a brief evaluation of it? Did their conclusions not follow the highly prescribed formula he had repeated time and time again?

The papers showed Steve what had really happened. The students didn’t know the content.

As much as they had tried to structure their writing, they just didn’t have enough subject knowledge. Steve expected a deep evaluation of quotes and he’d even taught the students how to go about discussing multiple interpretations of keywords and phrases.

But the students hadn’t memorised the quotes.

It got worse. The case studies in the exam were supposed to trigger students to consider socio-economic theories, court cases and historical events.

But the students only understood the ones they were tested on in class and so hadn’t read widely enough to answer the questions in the actual exam.

Why do your students fail?

Your students succeed and fail due to many factors. They may lack knowledge and understanding of a theory, method or event. They might not have ‘memorised’ the information they need. Their skills of analysis and evaluation may undermine the depth of their understanding. Steve considered all of these possibilities but was still at a loss to explain the underperformance. The truth was, that these weren’t the only factors that were at play. It’s often more complex.

Let’s look at why three particular students failed:

Student A had recently been dealing with a bereavement of a close family member. This had taken its toll on the student, who had performed well up to that point. In the final run-up to the exam, Steve had believed that this student would cope well with study leave, having demonstrated for almost two years that he could work well independently. However, in this instance he was wrong. The student was unable to focus at home, in the way he could at school, in part because he was constantly surrounded by distractions relating to the passing of his relative. Whilst his bereavement would not be much easier at school, at least he may have found some space to concentrate a little better, or for longer periods, enabling him to perform better than he eventually did on exam day.

Student B had a poor track record regarding her attendance. But despite this, she still managed to perform well in her assessed essays. As it turned out, she was close friends with a student who had written the same essays in the previous year. She re-worded these essays and in some cases had even memorised them by rote, for closed-book timed assessments in class. By doing so, she evaded the attention of staff who were actively looking for students requiring intervention. Since her grades were good, they didn’t consider her to be at risk of failing. Her problem though, was that in the exam she was not able to adapt those memorised answers when the question changed ever so slightly. She pulled the wool over many eyes, including Steve’s and failed outright.

Student C was a high performer. At GCSE she had achieved all A* and A grades and had done so with little visible effort. Throughout A Level, however, she had not always enjoyed the same level of success. Essay grades ranged from A* to C. Steve had been hot on the case with this student and had accurately identified where marks were being gained and lost. He gave thoroughly detailed feedback to the student, who was able to redraft the essays to an excellent standard, following the advice he gave. But on the day of the exam, her marks were inconsistent across the paper. Why had she performed so well in some areas, but so poorly in others? As it turned out, the detailed feedback had made no difference. Why? The student hadn’t had to think hard enough for herself as to how to improve. In the end, her highest marks came from the topics where Steve’s feedback was much more limited in detail (despite the formative essays being of an equally low quality to others where feedback was detailed). In this instance, the student had performed badly overall because she hadn’t become independent enough. She was still overly reliant on the teacher to help her to improve, even in the final weeks and days before the exam.

Action points for “Black Box” teachers

  • Assess regularly. Balance scheduled tests with unscheduled ones to accurately identify true levels of understanding.
  • Use rigorous assessment methods (past paper questions, etc)
  • Give feedback that strikes the right balance between being too detailed and not detailed enough
  • Create and maintain a ‘culture’ of student independence
  • Reward resilience and genuine effort, rather than high attainment alone
  • Test knowledge and understanding in creative ways, to avoid “scripted” responses
  • Formalise how you will act on the data you collect. Checklists are a time-efficient way of developing set procedures. (More on this in a future post!)

Final thoughts…

Learning from failure is sometimes the only way. I would love to hear your own stories of “Black Box” thinking. In the meantime, you can take a look here at Matthew Syed’s Black Box Thinking (my Amazon affiliate link).

Please leave a comment on this post or send me a tweet (@guruteaching). I’ll get right back to you!

Follow me on Twitter and now on Pinterest too. 

And don’t forget to LIKE and SHARE!

Disrupting Education

Disrupting Education

When should we rock the boat?

Disrupting education is essential to bring schools into the 21st Century. Why? Schools are old. Really old. The system isn’t much different now to how it was during the Industrial Revolution. The aim of schools back then was to create a population fit to work in a variety of factories. Subjects were taught in isolation from each other, with no differentiation. Today, we have differentiation, but largely the same model. Only now, we are moving people out of factories, as robots are moving into them. We aren’t needed in factories anymore! Schools need to address this urgently, or we will become irrelevant.

Why is disrupting education the solution?

Ask yourself: Do we maintain the status quo in education because we already have the best possible model? Or are we afraid (or just unwilling) to change the current model because the new one might not work?

I like the word “disruptive”. When something is disruptive it focuses our attention on it. We think about it properly and act on it. When was the last time we properly considered disrupting education, eg. how and why we plan our school buildings, timetables, technology and lessons the way we do? I think we should. I believe we should completely revolutionise what a school’s purpose is, who it is for and how it operates.

Last week when looking for something to read this summer I came across founder of Wired magazine and “Futurist” Kevin Kelly (@kevin2kelly). In his new book, The Inevitable, he discusses some of his ideas about how the future is likely to look in 10 to 30 years time.

We already know that there will be new devices that our lives will revolve around, like mobile phones today. These devices probably won’t even be invented until a few years time. We also know that Artificial Intelligence will be so sophisticated, cheap and widely accessible, that much of today’s “education” in schools will be largely redundant. The result? We won’t need to learn things that computers will do automatically and much more efficiently for us.

I always remember my Maths teacher telling me that I should put my calculator away because I’m not going to have one in my pocket at all times when I’m older. Ha! So, is new and emerging  new technology already making some of our subjects obsolete? Let’s see…

How do we use technology already?

1. Google Translate: can translate languages at the click of a mouse

In Modern Foreign Languages lessons, we learn how to spell, pronounce and understand different languages. We do this primarily because if we didn’t, then it would be difficult to communicate with others around the globe. What if technology removed that barrier? Would we still teach and learn other languages? Would there be any point? My heart is saying yes, but my head is saying no (for the vast majority of students). Would it become a luxury rather than a necessity to be able to speak fluent French in future?

2. Augmented Reality: apps can project holograms of the people you are speaking to on Skype

What if we didn’t teach students from our catchment area, but from around the world instead? Imagine each student, from countries around the world, sitting in your classroom (if a physical classroom is even needed), in holographic form. It sounds far-fetched, but there are companies all over the world who are already adopting this technology for holding meetings and training events. AR is already disrupting industry. It will disrupt education. I think that AR will become a feature of top performing “global schools” at some point in our lifetime. As teachers, we need to begin thinking about how we can best utilise this technology, although perhaps on a much smaller scale at first!

3. Virtual Reality: can simulate practical learning environments in standardised and measurable ways

In many industries where the work is physically demanding, involves an element of risk, or requires a detailed and standardised analysis of techniques used, we use simulators to train workers. These simulations used to take place in huge physical machines – think of the flight simulators you see at fairgrounds. Nowadays, simulators rely less on machinery and more on software to simulate the ‘sensation’ of our physical environment. Consider this: some schools in our inner cities have little in the way of outdoor space for athletics training. What if we could train students on a physical technique in a classroom environment, where a ‘virtual’ javelin was used instead of a physical one? Data could be gathered and analysed immediately using inbuilt software and could give personal feedback, in real time, to 30 students at once. No marking required!

Future Technology

My question to you:

If we could implement any of these (or other) technologies now, at no cost, how would it change the way you taught your subject?

Follow me on Twitter and now on Pinterest too. 

And don’t forget to click on the share buttons below!

%d bloggers like this: