School Inspections

#Ofsted's @mcladingbowl via @rachelala asks everyone to share a school inspection myth-buster. So here goes.

---

Ofsted is the Office for Standards in Education, Children’s Services and Skills, reporting directly to Parliament. It carries out hundreds of inspections and regulatory visits every week across England; separate bodies handle inspections in the rest of the UK.

According to Ofsted, inspection acts in a number of ways to drive and support school improvement.

• raises expectations by setting the standards of performance and effectiveness expected of schools
• provides a sharp challenge and the impetus to act where improvement is needed clearly identifies strengths and weaknesses
• recommends specific priorities for improvement for the school and, when appropriate, checks on and promotes subsequent progress
• promotes rigour in the way that schools evaluate their own performance, thereby enhancing their capacity to improve
• monitors the progress and performance of schools that are not yet good, and challenges and supports senior leaders, staff and those responsible for governance.

Of course, schools wish to perform well in these inspections, and to avoid the inconvenience and shame of being selected for "special measures". So it is not surprising that certain beliefs have grown up around these inspections.

Although I don't have direct experience of school inspections, I can easily imagine how such beliefs might develop. Perhaps the inspector visiting school A casually asks to see a particular document, or comments on its absence. This becomes part of the collective memory of the school. Before the next inspection, every teacher in the school has been instructed to prepare this document in readiness. In time, this knowledge spreads to other schools, and becomes widely accepted as "best practice" for passing an inspection.  

Ofsted now wishes to dispel certain myths about the inspection process, and denies that it "requires" or "expects to see" loads of stuff. In its latest clarification, Ofsted officially deprecates a number of specific practices.

But there is a critical ambiguity in the notion of requirement. These practices may not be officially required by Ofsted; but if they happen to be strongly correlated with successful inspections, it may well be rational for school teachers to continue to regard these practices as implicitly encouraged and reinforced by the actions of Ofsted inspectors. It would be a brave head teacher who abandoned those practices that had got the school through past inspections, simply because Ofsted insisted that these practices were not officially required.

Ofsted is also charged with a range of political and social objectives, including promoting British values and tackling extremism, and these implicit objectives are believed to colour its assessments of educational outcomes.

And when Ofsted insists that "it is up to schools themselves to determine their practices and for leadership teams to justify these on their own merits rather than by reference to the inspection handbook", this sounds suspiciously like a classic double bind. Of course you can have as much bureaucracy as you like, as long as you really want to do it for its own sake and not because we told you to. Oh, and please make sure none of it is "unnecessary".

---

Ofsted, Framework for School Inspection (July 2014)

Ofsted inspections - clarification for schools (October 2014)

Richard Adams, Ofsted tells teachers what not to do in effort to dispel inspection myths (Guardian 17 October 2014)

Graeme Paton, Ofsted being turned into a 'schoolroom security service' (Telegraph 28 November 2014)


Zoe Williams, Swamp or success: your school is being racially profiled (Guardian 23 November 2014)

Target-Setting

WYMIWYG - What You Measure Is What You Get

This is another very important system principle. Systems are distorted by the presence of targets, which increasingly fail to measure what they were supposed to measure. 

Here is a well-known example. If you test, measure and analyse the ability of school-children, this may tell you all sorts of useful things about the socio-geographic distribution of ability, about the success of different teaching methods, and so on. But if you set targets for the scores achieved on these tests, this will motivate some changes in behaviour among teachers, parents and children. Teachers will "teach to the test", while many middle-class parents will send their children for special coaching. (Indeed, for many schools, the excellent results achieved in the school league tables are in large part due to the huge amounts of extra tuition received by children outside school, and bears little relationship to the amount of added-value provided by the school itself.) While this may indeed improve the scores, it seriously undermines your ability to learn anything useful about the ability of school-children, or to make any systematic changes. 

Another interesting example came up in the UK election. Given a reasonable-sounding goal that health clinics should try to see patients within 48 hours, the government had imposed a target. It emerged (on live television, to the surprise and embarassment of the Prime Minister) that in order to reliably achieve this target, some clinics had changed their appointment policy and were now refusing to book appointments more than 48 hours in advance.

Jenni Russell When you can't see a GP (Guardian, April 30th, 2005)

The present Labour government is often criticized for its obsession with targets, but it should be remembered that this obsession was shared by the Conservative government under John Major. The desire to set measurable targets often comes from quite sincere motives, but these targets have dysfunctional effects. This is especially true when targets are set by politicians under political pressure without proper systems analysis.

• Setting isolated targets for improving the things you are unhappy about, while failing to set targets for maintaining the things you are happy about.
• Setting sample targets as illustrations of the things you could improve, which then receive disproportionate amounts of attention and resource. ("We will cut waiting times for breast cancer" ... hang on, what about other forms of cancer such as prostate?)

And of course target-setting is not just a disease of politicians. It is also an occupational hazard of managers within organizations. We cannot reasonably demand an end to targets, but we need to work on developing wiser targets. 

 

POSIWID - Purpose of System Is What It Does

At one level, POSIWID shows us how complex systems resist simplistic attempts to change them, or sometimes even to monitor them. The education system contains a testing subsystem whose purpose is to achieve high scores. Tracing how the testing subsystem actually achieves high scores reveals some important dependencies: the test results are dependent on the coaching subsystem, which in turn is dependent on the social system of the parents. 

If we want to make meaningful changes in the education system, it is very useful to carry out this kind of dependency analysis, because it helps us to predict several things:

1. How successful a given initiative will be.
2. How quickly this success will become visible.
3. How unequally this success may be distributed in different areas.
4. How quickly this success may be eroded or undermined by other system effects.

At another level, we can ask about the POSIWID of the target-setting system itself. The avowed purpose of target-setting is usually something to do with accountability or performance or both. The actual effect is often to make things more complicated and more bureaucratic, achieving local accountability or performance only at the expense of accountability and performance elsewhere in the larger system.

It is possible to intervene into this system too, but it needs an intervention at the right logical level. Perhaps incredibly, the quality standard ISO 9000 (dismissed by many people as hopelessly bureaucratic) contains a defined point where you can plug in a control loop that will limit or even reverse the growth of bureaucracy. I really enjoy that kind of intervention - it's like one of those martial arts where you simply redirect the energy of your opponent. 

 

See also On the performativity of data (August 2021). More posts on target-setting.