Better school improvement through Problem-Driven Iterative Adaptation

Guest blog written by Harry Fletcher-Wood

What do these have in common?

• Crossrail
• The Edinburgh Tram
• The NHS National Programme for IT

All three ran late and overbudget: Crossrail is 3 years late and £2 billion overbudget; the Edinburgh Tram was half-finished 3 years late and £231 million overbudget; the NHS IT programme was abandoned after 10 years and £6 billion. Most big projects exceed their budget – because when people make plans, they are usually optimistic, overconfident, and under pressure to promise success (Flyvbjerg et al., 2018, 11-12). Promising plans pose subsequent problems.

Which brings me to school improvement. How many things did we drop from the School Improvement Plan and professional development programme last year – because unanticipated challenges took precedence? This example is so extreme – so obvious – that it risks undermining my point. The same thing happens every year.  We finish our plan, and – almost immediately – new priorities arise, external pressures shift, colleagues move role. Much of the plan is never fulfilled.

I’ve come to view a long-term plan as a necessary fiction. It may all come off: if it does, I’ll believe in it. I don’t mean we shouldn’t plan or prepare. I mean we should recognise that our foresight is limited: unexpected events will make fulfilling the plan hard; new insights will make us change direction. Planning a series of steps (and expecting they’ll actually happen) prepares us poorly. In this post, I’ll explain why, and suggest a better way to plan school improvement, drawn from development economics.

The problem with big plans and best practice

Consider these events:

• 2000 – 2007: Major demand on Mozambique’s judicial sector. Insufficient and disorganised supply
• 2007 – 2012: Donors support multi-million dollar project to build case management system to improve management in sector
• 2012: Project monies have been disbursed. No system exists and Mozambique’s judicial sector is still poorly managed. There is no coordinated availability of data to make decisions, make budgets or manage resources.

(Andrews et al., 2015, 129)

When we face a problem, our instinct is often to create a grand plan to solve it: in this case, a multi-million dollar, internationally-supported project. We emulate best practice: we want a world-class case management system. For some challenges – like building schools – this can work. But it doesn’t work for complicated challenges rooted in human behaviour, such as getting multiple judicial agencies to cooperate, or ensuring students experience a great education.

Big plans pursuing best practice make organisations look good (helping them survive). But they neither help them meet their goals, nor build capacity to meet those goals:

Many school reform initiatives fail to achieve sustained improvements in performance because schools pretend to reform by changing what policies and organisational structures look like rather than what they actually do…

This process results from a pressure to mimic – when schools face ambiguous goals, are risk averse, uncertain about the means to achieve them and are dependent to varying degrees on external bodies.

– Andrews et Al., 2015, 123-124

This pressure to mimic other schools has many causes: Ofsted’s criteria, teachers’ and parents’ expectations, schools’ attempts to imitate successful peers (Allen and Sims, 2018, Ch.7).

This process can compound upon itself, eventually making failing and

flailing schools both internally resistant to reform and immune to external pressures for any real change: the more things change the more they stay the same.

– Andrews et Al., 2015, 124

The quoted text describes international development: I’ve replaced the word ‘states’ with ‘schools.’ Yet it rings true: schools face ambiguous goals and external pressure; they often copy what seems to have worked elsewhere.

I began this post by arguing that big plans are too inflexible to be helpful. Let me go further: big plans can be harmful. They fail to achieve their goals and they undermine teachers’ and schools’ capacity to improve.

A better way to make improvements

I’m not claiming we shouldn’t aspire to best practice, or learn from others. But what works elsewhere works due to time and experience, trial and error, teachers’ accumulated expertise. Consider these alternatives:

A) A big plan – We decide a priority for the year: helping students retain learning better. We choose an evidence-informed solution: building retrieval quizzing into lessons. We plan the required training, coaching, resources and performance management targets. Then [delete as applicable: a senior leader leaves/ Ofsted come/ new government guidelines] and so [delete as applicable: training is cancelled/ coaching focuses elsewhere/ resources prove unpopular]. At the end of the year some teachers are using retrieval practice effectively. Many are not. Next year, we move on. 

B) Iterative adaptation – We work with staff to identify a problem: students struggle to retain learning. We don’t choose a solution or plan the whole year – instead we form a working group (probably volunteers), and agree a first step (informed by the evidence): building retrieval quizzing into lessons. The volunteers try this for a month, then we review what’s happened and what we’ve learned. It seems to help, but it’s taking a lot of planning and lesson time. We agree a different approach: planning questions as a team and setting them for homework. We try this for a month: it works better, but checking answers still takes too much lesson time. We agree to move the quizzing online – this works well. After a term, we have developed a workable solution – and a group of experienced advocates for it – we can share with the rest of the staff.

What makes Problem-Driven Iterative Adaptation different?

The hallmarks of this approach – known as Problem-Driven Iterative Adaptation (PDIA) – are:

• Finding local solutions for local problems: don’t copy best practice, work with local teams to solve the problems which matter most to them
• Encourage positive deviance: encourage experimentation and find people who are solving the problem already
• Iterate and adapt: experiment, review, adapt
• Scale through diffusion: create champions who ensure the change is viable, legitimate and relevant (Andrews et al., 2015, 125)

Many aspects of this are familiar: involving staff, piloting change, gaining feedback. But I’m not sure how often we integrate all of them in school improvement. We talk more about making and delivering plans than we do about designing processes to support improvement. When we implement a plan, we try to achieve its goals. When we do PDIA, we solve our problem, perhaps changing our goals to do so: retrieval quizzing might not be the solution; we may need to revise our curriculum first.

PDIA works because it:

• Is simple: we agree one step at a time
• Is flexible: we don’t need to plan every step, we just need a broad direction; we learn and adapt as we go
• Permits (and encourages) changes of course as events unfold, and as we learn

PDIA also makes change easier and more attractive:

• We agree one small step, then review what happened
• Successful steps earn the support of those involved and the interest of those not yet involved
• We avoid having to argue over the perfect plan: we solve problems by acting and learning

Conclusion

A policy’s architect cannot anticipate and plan for every challenge and opportunity those implementing the policy will face. When we implement, we test a hypothesis (that the policy will help): we begin a process of exploration and evolution (Pressman and Wildavsky, 1984).

To make this process work, we must connect our goals and reality. Without a goal, and resolution to fulfil it, we are at the mercy of events. But determined pursuit of one goal can become increasingly unrealistic. Problem-Driven Iterative Adaptation bridges this gap: it lets us build capacity, solve problems that matter and learn from experience. It offers a better way to improve schools.

Further reading

The team advancing Problem-Driven Iterative Adaptation have shared their work generously. I particularly recommend:

• This paper: an accessible overview of the approach and its merits
• This fascinating (free) book extending the argument, by explaining how capability traps preventing development, and how they can be addressed.
• The PDIA Toolkit, which offers invaluable guidance in putting it into practice.

This blog first appeared on the Improving Teaching Blog.