The unintended consequences of pursuing an ‘outstanding’

Although I now have the luxury of not being accountable to Ofsted, for ten years as a primary headteacher I attempted to navigate the shifting sands of successive inspection frameworks, coming under pressure from various stakeholders to achieve the coveted (by some) ‘outstanding’ judgement.

Since Amanda Spielman assumed the role of Ofsted’s Chief Inspector in January of this year, I have been greatly encouraged and reassured by the tone and focus of her speeches and how this differs to her predecessor.

I especially warm to her suggestion that a review of the future of the ‘outstanding’ grade should be considered as part of the wider programme to review Ofsted’s work.

Furthermore, she highlighted the need to fully understand the way in which Ofsted’s grade structure influences the behaviour of schools.

In my view, the status attributed to achieving the ‘outstanding’ and the unintended consequences of pursuing this goal is an example of Goodhart’s law. In other words, when a measure becomes a target, it ceases to be an effective measure.

Good with outstanding features

This probably sounds like sour grapes, coming from someone who was head of schools which, under his tenure, were never judged to be better than ‘good with outstanding features’ in the old terminology.

However, I never really came to terms with the outstanding label, mainly because I knew of schools with this accolade that I would not choose to send my own child to.

I recall visiting one such school to play a football match and, as our contingent left afterwards, our cars were pelted with stones by some of the players from our ‘outstanding’ neighbours.

Clearly something had slipped through the ‘expectations for social behaviour’ net in this instance.

Training for Ofsted to interpret assessment data

The Ofsted process has attempted to measure many things, but there are numerous aspects of school life which receive little, or no, consideration under the inspection framework.

There are many unrecognised and almost intangible traits which make a difference to establishing a positive ethos and ensuring school effectiveness.

One such nebulous concept is that of assessment.

The word ‘assessment’ appears on a number of occasions throughout the inspection framework, but there are very few clues as to how the effectiveness of a school’s assessment procedures will be judged by inspectors.

It is therefore of interest to note that in Ofsted’s School inspection update; special edition (September 2017) explains that inspectors have been receiving training to help them understand the concepts of reliability and validity when interpreting school assessment data.

Recognising the importance of reliability and validity

As a newly enrolled student on Durham University’s MSc course in Educational Assessment a few years ago, I vividly recall the introductory talk by the course leaders. They explained that they would be delighted that, if nothing else, by the end of the course we clearly understood the principles of reliability and validity in the context of assessment.

Reliability and validity are the cornerstones of assessment theory and practice and, perhaps more importantly, how data is interpreted and how inferences are made from this information. The simplest things, such as where a child sits during a test, can compromise reliability and validity and everyone involved in educational measurement should have a sound appreciation of these key principles.

Inspectors undertaking this kind of training can only be a good thing. Interestingly, they are also to have the services of a data analyst helpdesk.

There are numerous reports in the educational press and on educational blogs which suggest that there is a very strong relationship between inspectors’ judgement of pupil achievement and the overall judgement a school receives.

High stakes

It is reassuring to know the Chief Inspector and her staff are saying such positive things, it is another matter for these directives to be embraced by those involved in conducting inspections on a day to day basis.

With so much at stake, the more that can be done to enhance inspectors’ ability to interpret data validly, the better.

‘Outstanding’ judgements should be based on more than data alone and interpretations must be made in the context of other supporting evidence. It was very pleasing to read in the government’s recently published findings from the Primary Assessment in England consultation that they now ‘recognise that no single piece of data or measure can be used alone to judge a school’s performance…’

Perhaps the tide has turned? Perhaps it is time too that teachers enhanced their own knowledge and understanding on the principles of what is arguably the most powerful tool in education: assessment.

Find out more

Read the research: ‘The unintended consequences of school inspection: the prevalence of inspection side-effects in Austria, the Czech Republic, England, Ireland, the Netherlands, Sweden, and Switzerland’

Read the Inspection data summary report (published 3 October 2017)