Skip to content
An official website of the OECD. Find out more
Log In | Register |
Created by the Public Governance Directorate

This website was created by the OECD Observatory of Public Sector Innovation (OPSI), part of the OECD Public Governance Directorate (GOV).

How to validate authenticity

Validation that this is an official OECD website can be found on the Innovative Government page of the corporate OECD website.

Seven Routes to Experimentation: A guide to applied behavioural science methods

Published on 20 September 2023

Introduction

“Seven Routes to Experimentation: A Policymaker’s Guide to Applied Behavioural Science Methods,” is an OECD resource created in collaboration with behavioural scientists at Affective Advisory and academics from the Swiss Federal Institute of Technology in Zürich (ETH Zürich). It offers guidelines and a visual roadmap to help policymakers choose the most fit-for-purpose evidence collection method for their specific policy challenge.

Source: Elaboration of the authors: Varazzani, C., Emmerling. T., Brusoni, S., Fontanesi, L., and Tuomaila, H., (2023), “Seven routes to experimentation: A guide to applied behavioural science methods,” OECD Working Papers on Public Governance, OECD Publishing, Paris. Note: The authors elaborated the map based on a previous map ideated, researched, and designed by Laura Castro Soto, Judith Wagner, and Torben Emmerling (sevenroutes.com).

The seven applied behavioural science methods:

  • Randomised Controlled Trials (RCTs) are experiments that can demonstrate a causal relationship between an intervention and an outcome, by randomly assigning individuals to an intervention group and a control group.
  • A/B testing tests two or more manipulations (such as variants of a webpage) to assess which performs better in terms of a specific goal or metric.
  • Difference-in-Difference is an experimental method that estimates the causal effect of an intervention by comparing changes in outcomes between an intervention group and a control group before and after the intervention.
  • Before-After studies assess the impact of an intervention or event by comparing outcomes or measurements before and after its occurrence, without a control group.
  • Longitudinal studies collect data from the same individuals or groups over an extended period to assess trends over time.
  • Correlational studies help to investigate the relationship between two or more variables to determine if they vary together (without implying causation).
  • Qualitative studies explore the underlying meanings and nuances of a phenomenon through interviews, focus group sessions, or other exploratory methods based on conversations and observations.

The choice of method is determined by these five key questions:

Source: Elaboration of the authors: Varazzani, C., Emmerling. T., Brusoni, S., Fontanesi, L., and Tuomaila, H., (2023), “Seven routes to experimentation: A guide to applied behavioural science methods,” OECD Working Papers on Public Governance, OECD Publishing, Paris.

Value added

The map offers a practical and user-friendly way for policymakers to navigate the diverse landscape of behavioural science methods. It supports:

  • Addressing a complex challenge: Policymakers often need evidence to support decision-making, but choosing the right method can be daunting due to personnel, time, and cost constraints. The map simplifies this process.
  • Standardisation: The lack of standardised methods and labels in behavioural science can be confusing. This map provides clarity by categorising and guiding the selection of widely used methods.
  • Fit-for-purpose evidence: This map supports a shift in applied behavioural science towards “fit-for-purpose” evidence, allowing policymakers to tailor their approach to the unique needs of their policy challenge.

It guides you to one or more methods that align with your needs. The map ensures a systematic and evidence-based approach to method selection, considering factors like generalisability, resource constraints, and causal inference.

The pros and cons of each of the seven methods are summarised below:

Source: Elaboration of the authors: Varazzani, C., Emmerling. T., Brusoni, S., Fontanesi, L., and Tuomaila, H., (2023), “Seven routes to experimentation: A guide to applied behavioural science methods,” OECD Working Papers on Public Governance, OECD Publishing, Paris.

How to use the map

  • Method selection: The map helps policymakers find a suitable evidence collection method to support their policymaking process.
  • Implementation guidance: The map offers insights into the requirements and considerations for each method, aiding practical implementation.
  • Communication tool: Policymakers can use the map to communicate the advantages and disadvantages of different methods to executives.

Based on your answers to the five key questions, you’ll be left with one or more alternative methods that align with your needs. This systematic approach ensures that you select the most appropriate method for your policy challenge, taking into account factors like generalisability, resource constraints, and the need for causal inference. You can use this map as a decision tool to pick a research method, as a pre-mortem tool to test your assumptions about the research project you picked, or as a way to communicate to executives about the trade-offs between research methods.

In summary, this map helps policymakers navigate the complex landscape of behavioural science methods by guiding them through a series of questions tailored to their specific policy challenges. It ensures a systematic and evidence-based approach to method selection, ultimately leading to more effective and informed policy decisions.

This paper is funded by the European Union. Its contents are the sole responsibility of the authors and do not necessarily reflect the views of the European Union.

PUBLICATION

Seven Routes to Experimentation: A guide to applied behavioural science methods

Published: 20 September, 2023

Download Report
Download Map