WYMIWYG - What You Measure Is What You Get
This is another very important system principle. Systems are distorted by the presence of targets, which increasingly fail to measure what they were supposed to measure.
Here is a well-known example. If you test, measure and analyse the ability of school-children, this may tell you all sorts of useful things about the socio-geographic distribution of ability, about the success of different teaching methods, and so on. But if you set targets for the scores achieved on these tests, this will motivate some changes in behaviour among teachers, parents and children. Teachers will "teach to the test", while many middle-class parents will send their children for special coaching. (Indeed, for many schools, the excellent results achieved in the school league tables are in large part due to the huge amounts of extra tuition received by children outside school, and bears little relationship to the amount of added-value provided by the school itself.) While this may indeed improve the scores, it seriously undermines your ability to learn anything useful about the ability of school-children, or to make any systematic changes.
Another interesting example came up in the UK election. Given a reasonable-sounding goal that health clinics should try to see patients within 48 hours, the government had imposed a target. It emerged (on live television, to the surprise and embarassment of the Prime Minister) that in order to reliably achieve this target, some clinics had changed their appointment policy and were now refusing to book appointments more than 48 hours in advance.
The present Labour government is often criticized for its obsession with targets, but it should be remembered that this obsession was shared by the Conservative government under John Major. The desire to set measurable targets often comes from quite sincere motives, but these targets have dysfunctional effects. This is especially true when targets are set by politicians under political pressure without proper systems analysis.
- Setting isolated targets for improving the things you are unhappy about, while failing to set targets for maintaining the things you are happy about.
- Setting sample targets as illustrations of the things you could improve, which then receive disproportionate amounts of attention and resource. ("We will cut waiting times for breast cancer" ... hang on, what about other forms of cancer such as prostate?)
And of course target-setting is not just a disease of politicians. It is also an occupational hazard of managers within organizations. We cannot reasonably demand an end to targets, but we need to work on developing wiser targets.
POSIWID - Purpose of System Is What It Does
At one level, POSIWID shows us how complex systems resist simplistic attempts to change them, or sometimes even to monitor them. The education system contains a testing subsystem whose purpose is to achieve high scores. Tracing how the testing subsystem actually achieves high scores reveals some important dependencies: the test results are dependent on the coaching subsystem, which in turn is dependent on the social system of the parents.
If we want to make meaningful changes in the education system, it is very useful to carry out this kind of dependency analysis, because it helps us to predict several things:
- How successful a given initiative will be.
- How quickly this success will become visible.
- How unequally this success may be distributed in different areas.
- How quickly this success may be eroded or undermined by other system effects.
At another level, we can ask about the POSIWID of the target-setting system itself. The avowed purpose of target-setting is usually something to do with accountability or performance or both. The actual effect is often to make things more complicated and more bureaucratic, achieving local accountability or performance only at the expense of accountability and performance elsewhere in the larger system.
It is possible to intervene into this system too, but it needs an intervention at the right logical level. Perhaps incredibly, the quality standard ISO 9000 (dismissed by many people as hopelessly bureaucratic) contains a defined point where you can plug in a control loop that will limit or even reverse the growth of bureaucracy. I really enjoy that kind of intervention - it's like one of those martial arts where you simply redirect the energy of your opponent.
See also On the performativity of data (August 2021). More posts on target-setting.
