The Evaluation Task Force - Observatory of Public Sector Innovation

The Evaluation Task Force (ETF) ensures evidence and evaluation sits at the heart of UK government spending decisions - any area routinely highlighted for improvement. ETF activity drives continuous improvements in the way programmes are evaluated to inform decisions on whether they should be continued, expanded, modified or stopped. It brings the same approach to the social sciences as medicine uses, fostering a culture of experimentation, learning and rigour.

Innovation Summary

Innovation Overview

In 2019, the Prime Minister’s Implementation Unit found that only 8% of the UK Government's £432 billion spending on major projects had robust impact evaluation plans in place. The average UK taxpayer was contributing £12,000 to programmes which had no plans to evaluate whether they were delivering their desired outcomes. Government wanted to try setting up a team of experts to address this.

The Evaluation Task Force (ETF) is an innovative unit, created in response to this issue as part of the UK government’s landmark Declaration on Government Reform. As a joint Cabinet Office-HM Treasury unit, the ETF’s objective is to put evaluation evidence at the heart of government decisions, so that HM Government (HMG) can have confidence the money it spends is delivering better outcomes for the British public, and delivers value for money. The ETF aims to ensure that government programmes put robust evaluations in place to learn lessons and continually improve outcomes for the public.

The ETF apply the methods of evaluation and experimentation that are common in medicine to the £1 trillion that HM Government spends each year on public services. We would never consider rolling out a vaccine that had not been rigorously tested and the ETF is trying to extend that approach to all important policies and programmes.

The ETF has a headcount of 15 employees and delivers a range of activities to tackle the main barriers to robust evaluations in government and foster a culture of evaluation and experimentation. This includes providing advice and support to (i) HM Treasury Spending Teams on the evidence and evaluation plans underpinning departments’ spending proposals, and (ii) departments on how to design and deliver robust impact evaluation.

During the 2021 Spending Review, we reviewed 86 bids worth over £10 billion in three weeks. The team worked with HM Treasury to consider the quality of evidence underpinning the spending proposals; check whether proposals used counterfactuals to test if their idea worked; and set evaluation conditions in departments’ spending settlements to improve the quality of evaluation for funded programmes. This is driving a change in the very culture of government. The team is actively monitoring the delivery of funded interventions to ensure they are rigorously evaluated and to inform future spending decisions, so the most important areas to government are the best understood. ETF activities will improve the quality and breadth of evidence produced ahead of future spending choices, ensuring government decisions can be based on what works.

In our first year of operation, we organised and delivered the Policy that Works conference. HMG’s biggest analytical conference to date (with more than 2,200 registrants from across the Civil Service), the conference highlighted available evaluation support services and best-practice across three days. High-profile speakers included three Ministers, an HM Treasury Director General, and the Government Chief Scientific Adviser.

As secretariat for the UK government’s network of What Works Centres, the Evaluation Task Force actively supports the generation, collation and translation of evidence in key areas for government & practitioners alike.

Our work is increasingly embedded in existing government processes. For instance, we work with HM Treasury to ensure that departments’ Outcome Delivery Plans include the appropriate consideration of evaluation evidence, robust evaluation plans, and commit to publishing evaluations in a timely manner. We also advise on appropriate evaluation plans via a number of advisory and decision-making bodies. This means we are part of the system and infrastructure when key decisions are made about programmes.

We are designing a cross-government Evaluation Registry which will make evaluation evidence produced by departments more accessible. This will help to ensure evidence from published evaluations is used to inform decision-making across government. Increasing transparency of government evaluations will also improve public accountability and widen access to lessons learned from previous policies.

Finally, we secured £15 million from HM Treasury to develop and deliver the Evaluation Accelerator Fund. The fund seeks to enhance evaluation activity and the creation of actionable evidence in HMG priority areas. By providing robust evidence of financial or efficiency savings from new interventions and innovative approaches to service delivery, the fund will fill existing evidence gaps and inform future spending decisions. To date, the fund has made awards to 16 projects, One successful bid will see wastewater in prisons monitored and tested for illegal substances, making it easier for prison staff to identify who is holding supply and reduce drug abuse in jails. Another will assess the effectiveness of a one-off payment to 18-year-olds leaving care on homelessness, employment and their involvement in criminal activity.

Innovation Description

What Makes Your Project Innovative?

The ETF is unique in that it is staffed almost exclusively by senior evaluation specialists. Its structure was informed by international lessons learned and best-practice evidence on improving public policy evaluation.

While the US Evidence Act gives agencies the authority to prioritise funding impact evaluations, it does not integrate this with budget processes or require any impact evaluation to take place as a condition of receiving programme funding. The ETF’s status as a joint Cabinet Office-HM Treasury unit, and its integration into Treasury processes, gives it leverage to improve the quality and quantity of evaluation across government.

The Commission on Evidence-Based Policymaking found that the ideal evidence infrastructure focuses on creating evidence-support systems as well as an evidence-implementation system. The ETF partners with What Works Centres to ensure that evaluation results are accessible and acted upon within government.

What is the current status of your innovation?

The Evaluation Task Force supports the delivery of priority evaluations across government. We work closely with analysts and policy colleagues to help design and deliver robust evaluations in priority areas and challenge them in areas where plans are limited. We have developed close relationships with departments and Treasury spending teams, offering targeted advice and support. We are delivering the Evaluation Accelerator Fund, a £15m programme that has so far supported 16 projects which will fill evidence gaps by testing and evaluating new policies. We work closely with colleagues across HM Government to ensure decision-making is informed by evaluation evidence and that effective systems are in place to deliver, manage and use evaluation evidence. We are actively promoting improved transparency of evaluation findings, and engaging teams across government to improve the way separate parts of the evaluation ecosystem work.

Innovation Development

Collaborations & Partnerships

The network of What Works Centres are strategic research partners for many government departments, offering deep subject matter expertise and links to practitioners that are piloting new approaches with potential to scale and inform policy elsewhere.

Our Oversight Board consists of senior leaders from across government. They help our team prioritise our activities and provide strategic support and direction when we face challenges and obstacles.

Users, Stakeholders & Beneficiaries

HM Treasury officials can access advice and support on evidence and evaluation plans which underpin spending commitments, and can do so on an ad hoc basis or over the longer term.
What Works Centres benefit from our team’s access and insights into department priorities.
Departments are challenged to cite evidence (or demonstrate robust plans for generating it) when asking for funding.
Taxpayers stand to benefit from increased transparency of evaluation findings.

Innovation Reflections

Results, Outcomes & Impacts

Since its inception, the Evaluation Task Force has advised on over 160 programmes covering £48 billion of spend.

During the 2021 Spending Review, we reviewed 86 bids, worth in excess of £10 billion, over three weeks. This informed a range of spending decisions in high priority areas, including youth services, criminal justice, education and COVID-19 recovery. We introduced evaluation-specific conditions into departments’ spending settlements for the first time. As a result, departments have published comprehensive evaluation strategies outlining a long-term approach to monitoring and evaluating their programmes.

HM Treasury’s Director of Public Spending shared that our involvement was critical to the success of the Spending Review, saying, “Because of the Evaluation Task Force, my team can make better critical choices on where limited public money goes, are more aware about evaluation options for policy areas, and can challenge departments to be bold and better on evaluation.”

Challenges and Failures

With only 15 members of staff, demand for the Evaluation Task Force’s advice and support exceeds what it is able to deliver, and so the team must target its engagement with departments and programmes. We currently prioritise programmes and the support we can offer their corresponding evaluations on the basis of programme cost, evaluation complexity, evaluation resources, and the current capability of the department conducting the evaluation. Areas with a limited existing evidence base are also prioritised.

Evaluation-specific spending settlement conditions are a key tool used by the team to push for more and better evaluation in departments. However, conditions can only be agreed during spending reviews, and these are typically every few years. The Evaluation Task Force therefore relies on support from its ministers and Oversight Board to agree and communicate new evaluation priorities outside of spending reviews.

Conditions for Success

Developing and maintaining good working relationships with lead analysts in departments has been very important. Our team needs to balance offering advice and support to departments with the need to support HM Treasury functions of objectively assessing the value for money of programmes.

Our team has also benefited significantly from the support of Ministers and senior officials in government who have previously supported and championed evaluation and evidence-based policy. Many senior officials serve on our Oversight Board or regularly meet with our director to provide advice and strategic insight.

The existence of the What Works Network has also been extremely useful in identifying evidence gaps and priorities in departments and making the case for the value of robust impact evaluations.

Replication

The Evaluation Task Force has only existed since Spring 2021, and to our knowledge it has not been replicated to date.

We have received a few foreign delegations to share reflections on our work, and through our website and webinars, we are also sharing regular updates on our work within the UK Civil Service and broader evaluation community.

We hope that publishing our strategy, outcomes, and impacts – as well as updates on our key activities and priorities – will aid other governments that are interested in creating evidence and evaluation units.

The ETF is also building the evidence of our own impact to help make the case that - as a new and innovative function - we should be scaled up in the future to ensure we can advise on a wider range of programmes.

Lessons Learned

Funding for evaluation is important, but it is not necessarily the key limiting factor or barrier to producing and publishing robust evaluation evidence.

Working culture and norms matter a great deal. There can understandably be a reluctance to test flagship policies or publish evaluation findings if there are reputational or professional consequences when programmes don’t work.

Agreeing professional standards around evaluation is a useful starting point, but this is not sufficient by itself. For instance, very few government-led evaluations follow existing standards when it comes to publishing their evaluation plans, despite this being an agreed standard within the Government Social Research profession. These issues drive our work around transparency and the development of an evaluation registry.