Decreasing corruption in the Colombian School Meals Programme PAE – A bottom-up approach

Call for Innovations Call for Innovations
This case was submitted as part of the Call for Innovations, an annual partnership initiative between OPSI and the UAE Mohammed Bin Rashid Center for Government Innovation (MBRCGI)

The Colombian Public Innovation Team ran a trial aimed to improve the quality and quantity of food served in the Colombian government’s school meals programme (PAE). It combined SMS messages to encourage parental engagement in the programme and lighter audits by a third party (local university students). Behavioural insights were applied to inform message design. The project included a learning phase to test and adapt elements of the intervention and an experimental evaluation (RCT) in the territories of Nariño and Cesar.

Innovation Summary

Innovation Overview

Tackling the corruption that affects the provision of school meals is a priority for the Colombian government. Since a video in 2016 revealed the extent of fraud and deception, the issue has become a national scandal. The video showed a provider taking photos of children with one same plate full of food, as supposed evidence that the food was being provided, but also the reality that , immediately after the photos were taken, children were actually receiving much smaller portions of food, directly into their hands.

The project involved a collaboration between several units in the Colombian government (National Department of Planning, Secretariat for Transparency, Ministry of Education, and Ministry for ICTs), the UK's Behavioural Insights Team and the Inter-American Development Bank, with the support of local government entities and universities. We followed the TEST (target, explore, solution, trial) methodology proposed by the Behavioural Insights Team (BIT). In the target and explore phases, we conducted field work (direct observations, focus groups, semi-structured interviews) with members of local school communities in Colombia (government officials, principals, teachers, students, parents, food providers, cooks) and found that (a) parents usually had little knowledge about the food programme and the spaces availaible for their participation, and that (b) the vast majority of parents, even in remote rural areas, had access to cell-phones with SMS capabilities.

Text messages have proved to be an effective way of informing and engaging individuals in a variety of contexts, including parents of school children. This inspired us to test an SMS component in the intervention.

The implementation of the intervention was divided into two phases. In the first, or "learning" phase, we focused on understanding how to communicate with parents to increase their engagement with PAE. The second phtase tested the approach at scale with an RCT. The intervention combined SMS messages, based on results from the first stage, with spot-checks conducted at the schools. We tested whether the intervention improved the quantity and the quality of the food.

In the learning phase, we gained knowledge about logistical needs and tested messages sent via a two-way SMS platform, i.e. parents were able to respond to the messages we sent, and receive back an automated response. We used the Ministry of Technology’s SMS platform Urna de Cristal. We texted parents information on the entitlements of their children within PAE e.g. "Hola Manuel, Hoy Daniel debio haber comido pollo, arroz y brocoli. Si comio eso?" For 5 weeks approximately, 3000 parents in 8 schools in Nariño (Southwest Colombia) and Cesar (Northern Colombia) received our messages. Parents were randomly allocated each week to see a variation in the content of the message, or to receive it at a different time of day. This allocation allowed us to test the impact such variations on the likelihood of getting a response from parents.

The messages followed an adaptive or “winner stays on” design. Each week we adapted the messages based on what we found to be the most successful variation the week before. Specifically, we tested the impact of the following on parents’ level of engagement: 1) to what extent personalisation of the message matters; 2) to what extent timing of the message during the day matters; 3) whether response rates are higher if we provide parents with more or less information about what their child should have received; 4) whether parents are more likely to respond to a closed- or open-form question.

After the learning phase, we deployed an RCT with a sample of 208 schools, randomizing at the school level. The intervention showed no effect on the primary outcome measures: quantity and quality of food supplied by providers (as assessed during the data collection audits) and parents’ satisfaction with the food served (data collected through a household survey).

We knew before the launch that, given the experimental design, the minimum detectable effect size was relatively ambitious. Considering the high cost of the intervention, expecting or requiring such an effect size for the intervention judged to be a ‘success’ seemed reasonable. However, the confidence intervals are not very wide, suggesting that the actual impact of the treatment was close to 0. Apart from the intervention genuinely not having an impact on provider behaviour, we think there might be 2 reasons leading to spillover effects and thus a smaller effect size:
-Baseline survey as treatment conducted by the university students in both Treatment and Control schools (providers adjusted their behaviour in response if they interpreted these as a sign of increased oversight).
-Providers adjust behaviour across the board, since they served both Treatment and Control schools, they improved food delivery in both.

Innovation Description

Innovation Development

Innovation Reflections

Project Pitch

Supporting Videos

Leave a Reply

Your email address will not be published. Required fields are marked *

Status:

  • Diffusing Lessons - using what was learnt to inform other projects and understanding how the innovation can be applied in other ways

Innovation provided by:

Join our community:

It only takes a few minutes to complete the form and share your project.