Speak up via WhatsApp: A Qualitative WhatsApp Survey of Syrian Refugees and Lebanese Host Communities
This case was submitted as part of the Call for Innovations, an annual partnership initiative between OPSI and the UAE Mohammed Bin Rashid Center for Government Innovation (MBRCGI)
The United Nations Development Programme Lebanon used WhatsApp to conduct qualitiative surveys to listen to Syrian refugees in Lebanon dealing with conflicts with the local Lebanese community. Using Whatsapp is an effective tool for collecting qualitative data from vulnerable communities at scale. WhatsApp is widely used, with 84% of refugee households in Lebanon on WhatsApp. WhatsApp's voice message function allowed us to send survey questions as voice messages and collect people’s stories directly including from people who are illiterate. 1036 people participated in our survey showing its success as a bottom-up, people-driven method.
Seven years into the Syrian crisis and with almost a million Syrian refugees residing in Lebanon, a country of 6 million inhabitants, surveys show that fatigue is rising in host communities. In 2014, when there was still hope that the displacement crisis would be temporary, 40 percent of Lebanese said that there were no tensions with Syrians. By 2018, that number had dropped to 2 percent.
What exactly drives Lebanese-Syrian tensions and how can conflict be prevented? In the absence of rich qualitative data from the people themselves, it is left to the media to shape that discourse; and that media narrative increasingly uses blunt stereotypes and pits one supposedly homogeneous Lebanese community against an equally homogeneous Syrian community.
Our WhatsApp tool ‘Speak Your Mind’ aims to address this data gap by more effectively harnessing Lebanon’s vibrant social-mediascape. Digital literacy is widespread among both host communities and refugees in Lebanon: 84 percent of refugee households use WhatsApp (VASyR 2017) and younger people in particular consider information relayed through WhatsApp as more trustworthy than traditional media. Tapping into these digital possibilities helps UNDP Lebanon have a more egalitarian relationship with people on the ground and collect real-time, localized data to strengthen its conflict-prevention systems.
The desired outcome is that Lebanon will remain stable and refugees will be safe through a crisis response that proactively responds to people’s fears and needs and detects early warning signs to prevent conflict.
We piloted our first WhatsApp survey in November 2017 in the village of Qaraoun in the Bekaa region, which hosts the highest number of Syrian refugees in Lebanon. The WhatsApp tool was developed together with target users, using a design thinking approach, during a one-day workshop. We then sent the survey questions as voice messages to mobile phone numbers registered in the area. Participants were asked to tell a story recorded as audio WhatsApp messages responding to questions on community needs, conflict dynamics and feedback on stabilization projects in the area.
The voice message option helped us to reach people who struggle with literacy. The survey ran for a whole month, with a new topic starting each week. A total of 242 people participated in the first survey, and 794 people participated in a second WhatsApp survey that we implemented in Bar Elias (also in the Bekaa) in March 2018.
The unique contribution of WhatsApp surveying is that it makes the collection of a large qualitative sample possible in a very short time. The narrative data we received was very rich, giving us a data volume comparable to conducting 1036 qualitative interviews (which would take months if not years). The story data added much nuance to our understanding of host community/refugee relationships in tense and vulnerable areas. It also shed light on some ‘hidden barriers’ to stabilization projects for vulnerable populations in the area, which will help us to refine these projects in the future. Crucially, our respondents gave positive feedback on the tool itself, saying it was an easy way to communicate their needs and concerns to international organizations.
We have now integrated the WhatsApp deep-dive qualitative survey tool as part of our regular early warning and tension analysis. Going forward, our objective is to use WhatsApp more broadly to support research, programming and coordination in the humanitarian and development sector in at least three ways:
1) Human-Centred Design (HCD) Tool: HCD is a problem-solving strategy that incorporates the needs, feedback and suggestions of end users of services at every stage of the design process. As an HCD tool, WhatsApp can facilitate consultation, prioritization, ideation, and prototyping with end users to ensure that our services are accessible and relevant for people on the ground.
2) Real-Time Monitoring Tool: WhatsApp can also support programming as a ‘real time monitoring’ tool by collecting input and feedback from beneficiaries and the wider community before, during and after project implementation. As such, the tool helps to remove barriers to inclusivity, facilitates ‘on the spot’ adjustments to programming and measures the impact of our interventions.
3) Cross-Country Communication Tool: The tool also allows for continued contact with respondents after they cross borders. Even when people change phone numbers as they move to new countries, they often maintain their WhatsApp contacts either by linking their existing WhatsApp account to a new phone number or by continuing it on the previous number. Such cross-country communication could help protection actors to better understand the protection needs of refugees who returned or resettled. For instance, the tool could offer a means of communication with Syrian refugees after they leave Lebanon.
What Makes Your Project Innovative?
The project is innovative, because it taps into people's everyday communication habits to produce better, people-generated data.
Rather than inventing a new app or form of communication, we tried to pick up people where they are by using an application that already forms an integral part of their everyday life. This made communication more natural and may explain why many refugees felt sufficiently comfortable to talk about sensitive issues such as army raids, detention and harassment. From our side, we also tried to shed the image of an overly technical and removed bureaucracy by personalizing our voice messages to foster two-way communication between people on the ground and international organizations. By more effectively integrating WhatsApp in our research, programming and coordination, we can enhance the legitimacy and effectiveness of development work by systematically including people's perspectives in a cost-effective (WhatsApp is free) and prompt manner.
What is the current status of your innovation?
We have finished the two survey pilots and are currently finalizing the project outputs. We are now trying to scale up by securing more funding for multi-sector, inter-agency WhatsApp facilitated community engagements. We are also trying to promote the survey tool more broadly as a research and programming tool through broader outreach with other international organizations, civil society and academia.
Collaborations & Partnerships
Lebanese host communities and Syrian refugees were key as their participation produced the survey design and the data. Local civil society organizations and community stakeholders facilitated the implementation on the ground through community outreach, phone number collection and conflict sensitivity advice. Other UN agencies supported the initiative through advice and data. The Lebanese government supported the initiative and has expressed interest in WhatsApp as a citizen engagement tool.
Users, Stakeholders & Beneficiaries
The survey helped to amplify the voices of very vulnerable communities whose stories are not usually
heard by using a form of communication that they are comfortable with. For international organizations, civil society and the government, the tool enhances the relevance and effectiveness of their services (through citizens' input) thereby making them more accessible and legitimate.
Results, Outcomes & Impacts
The WhatsApp survey tool has added much nuance to our understanding of host-community/refugee tensions in Lebanon, thereby helping us to upgrade our conflict prevention system. The surveys have also produced deeper insights about the needs, fears and safety concerns of Syrian refugees and Lebanese host communities which will support humanitarian and development programming in Lebanon. For example, the survey found that a solar lighting project alongside a community promenade was less accessible to Syrians than assumed due to a de facto curfew in place that made it difficult for men to move after dark. Better understanding such hidden barriers helps to design projects that are equally accessible to Lebanese and Syrians in the future. The data also showed that communication around assistance needs to be improved as the criteria for assistance are often not clear to refugees which fuels suspicion and tensions between Syrian refugees. Another concern was that the supermarkets for which refugees get food vouchers are more expensive and often further away than more local supermarkets – an issue we have raised. The data will be published as a conflict analysis report in the next couple of weeks that will help UNDP and its partners in Lebanon to design their peacebuilding interventions in a way that is conflict sensitive.
Our WhatsApp participants told us that this was a much needed tool to create direct two-way communication between vulnerable communities and international organizations. The innovation project also yielded broader impact within the organization. Our programming colleagues from UNDP are now piloting the WhatsApp survey as a real-time monitoring tool for their livelihoods and peacebuilding programming with youth. We are publishing two research reports (one was already downloaded 795 times), one WhatsApp surveying guide and one podcast to promote the wider impact and usage of the WhatsApp tool.
Challenges and Failures
One key shortcoming of the WhatsApp survey was an underrepresentation of women. Women only represented one third of our respondents. Among refugees, female participation is lower as women have less access to the household phone. Their participation can be increased by varying the times at which questions are sent and running surveys for longer time periods to ensure women have more chances to reply to questions. Yet, lower female participation was not only a function of restricted mobile phone access but also reflected an underlying assumption that our survey questions were addressed to men. Some women would write to us that their husbands are currently at work and would reply to our question when they come home in the evening. For future surveys, we would suggest labelling a few questions as ‘for women only’ to make sure men hand over the phone to female household members.
Conditions for Success
Community mobilization and outreach are key for co-designing the survey, building the survey's credibility and ensuring conflict sensitivity. In both pilot locations, we organized a community workshop that also served as a platform for community mobilization and outreach. One key obstacle in terms of building the survey’s credibility was that people are not used to the United Nations communicating through such an informal application and thus suspected the survey to be spam or fraud. We encouraged community stakeholders and refugee focal points to promote participation in the survey especially among younger people and vulnerable communities. They received flyers and posters to distribute. We also put up posters in the municipality and in the Informal Settlements (IS) announcing the survey and asking people to save our phone number to receive the survey questions.
Support by senior leadership is key for mainstreaming the WhatsApp tool more broadly within the organization.
There is great potential for the replicability of this innovation project given the popularity of WhatsApp in most development and humanitarian contexts. We have already been approached by other UN agencies in Lebanon as well as from other countries to advise them on the implementation of such a survey. To faciliate replicability, we have devised a WhatsApp surveying guide which lays out the key steps and practical considerations involved in WhatsApp surveying and reflects on our lessons learnt. The WhatsApp surveying guide will be published shortly on UNDP Lebanon's website.
In our experience, WhatsApp surveying works best as a deep-dive qualitative tool to better understand particular localities or demographics. The ideal sample size is between 2000 and 3000 numbers which should give you between 340 and 510 respondents. Our response rate in both surveys was 17%. Our pilots suggest that the response rate varies substantially depending both on the question as well as the sending and follow-up strategy. We found that the response rate can be significantly increased through individual follow-up via WhatsApp and SMS messages. Calling respondents (which we did not do) could also increase the response rate. Going forward, WhatsApp business, a new App developed by WhatsApp, can boost the survey’s credibility and thus the response rate as it verifies the account of ‘businesses’ (in our case UNDP) and allows for automated messages and better organization of data.
Collecting phone numbers through local stakeholders: Since we did not have access to a database of phone numbers, we had to collect the numbers ourselves. In Qaraoun, the municipality supplied us with the phone numbers of Lebanese and Syrians registered in the village. In Bar Elias, local informants helped us to collect phone numbers through Shawishes (local camp managers), local NGOs, professional and private networks. Such collection was facilitated by the fact that much information sharing and organization around refugees and aid distribution already works through WhatsApp. Using different sources to collect phone numbers helps to create better, more representative samples. That said, the best route to representative sampling would be by accessing phone numbers directly through a data sharing agreement with a phone company. The phone company could then supply a database of phone numbers stripped off all personal identification (other than locality of phone registration, if the survey is focused on a particular area) which could be then used for administering the survey.
Ethics: There is a risk that such a WhatsApp survey raises expectations of concrete assistance among people who struggle to survive on an everyday basis. We tried to manage these expectations through a carefully worded introductory message that explained the purpose of the study. However, some of the messages we received showed that it is almost impossible not to raise expectations when engaging with very vulnerable communities. We compensated survey participants with phone credit for their time and data use at the end of the survey and with the hope that this may somewhat offset the frustration with research work that does not provide any tangible benefits.
Data Safety: We ensured data security by holding as little personal data as possible in the first place. Once the phone numbers entered our data systems, they were already stripped of any personal information (e.g. names). We also advised people not to give out any personal information (e.g. names, ID numbers) in their messages.