CODEPLAN’s project of hearing the people using government installed Call Center infrastructure and scientific survey research techniques is the first of its kind in Brazil and Latin America. Its major difference from prior uses of Government Call Centers and Ombudsman services is that we actively seek the populations’ opinions about specific topics, instead of passively waiting for their criticism and demands.

Traditionally, Ombudsman services receive criticism, requests, and suggestions and distribute these to the responsible government agencies. This is, no doubt, a very important use of government Call Centers. Still, such services have biases towards over-representing those more inclined to contact the government. Certainly, higher educational levels and income, on one hand, or populations from deprived areas on another, could probably be more inclined to contact Ombudsman services. These groups are not necessarily representative of the entire population in a city. Our project improves the use of Government Call Centers by actively seeking the opinions and evaluations of representative samples of the population, covering the entire territory and avoiding selection bias.

Given the overwhelming increase in the coverage of mobile and landlines in the Federal District and that 97% of the population lives in urban areas, we are able to reach a much broader audience, avoiding hearing only subsets of the population with specific inclinations. In addition, we utilize questionnaire design techniques that further limit biases on question wording or order, aiming at capturing interviewees’ true and honest opinions. We follow research ethics procedures, assuring anonymity and voluntary participation in the surveys, which also increase data reliability. Hence, the quality of data, its reliability, validity, and representativeness are trademarks of our project, advancing upon other sources of data collection in the government, especially about how citizen’s think.

The project “Hearing the People” aims at increasing the government’s levels of information about popular evaluations of public policies and services. The main problem is to reduce the distance between representatives/bureaucrats and citizens through the collection of reliable, valid, and representative data on citizen’s opinions and views about the quality of service provision. Through this mechanism, we aim at assisting other governmental institutions in improving the quality of public services and policies, accelerating the feedback loop in the policy cycle.

The problem of low-quality services and policies is rampant in the developing world, due, in part, to governments’ inability to consult the population about their demands and perspectives. The disconnect between decision-makers and population has become even more flagrant in Brazil after the massive public demonstrations in 2013, in the wake of the Confederations Cup. Bridging this gap is imperative to improve the quality, efficiency, effectiveness, and efficacy of public policies and services. The traditional mechanisms to hear the people, such as direct forms of democracy including public hearings, committees, referendums, and plebiscites are expensive, time-consuming, and rare. Not to mention that they may suffer from serious selection bias: those who chose to participate are not representative of the entire population.

A way around this is to contract private sector polling firms to conduct household, face-to-face or telephone surveys. Still, this requires additional expenditures and contracts, making the process costly and intermittent. To overcome the limitations of public consultation, we identified the existence of the Telephone Call Center infrastructure in the Federal District Government and, using our knowledge on sampling and questionnaire design developed the innovative idea of employing this installed equipment to hear the people efficiently, frequently, and scientifically. We can conduct around 4000 interviews in a period of 10 working days and produce a report in under 30 days, from the first meeting with our partner to the publication of the final report.

We produce expedite, reliable and valid informational feedback to governmental agencies. None of the existing forms of direct, participatory democracy possesses the same capability. We, therefore, maximized the utility of the Call Center, optimizing its use to evaluate public policies and services from the people’s perspective. The Government’s Call Center contains 178 operating posts used to operationalize several distinct services for the population. We use it in the off-hours to conduct computer-assisted telephone interviewing of the population. Hence, other services are maintained. We put the existing contract to its maximum use. Our business model is simple, with little bureaucracy.

Our partners show interest in conducting the policy or service evaluation through a formal letter. We organize meetings to define the content of the questionnaire, with intensive participation of the partners. In parallel, we select the sample using scientific calculations. Subsequently, telemarketing professionals at the call center receive training to conduct interviews with the selected sample. Finally, we elaborate and present the final report based on the data collected. This is the traditional model of survey data collection and analysis used by polling firms and academia. We employ these well-known techniques to answer governmental agencies’ questions about evaluations of public services.

We have evaluated the performance of the project by requesting our partners to assess the quality of the data collected and its impact on their service provision. Feedback indicates that the data was useful to reevaluate the offering of some services, improve agencies' websites and communication strategies, rearrange organizations' lists of priorities in attending clients’ requests, and assisting in reconsidering the policy proposals of government institutions. In spite of the diverse uses, all concur that the quality of the data and the rigorousness of collection methods are undeniable.

We are now at the moment of diffusing lessons about our innovation. One such way is participating in dissemination activities such as enlisting our project in initiatives such as the Edge of Government. We have been classified as a finalist in Brazil’s National Public Administration School (ENAP) Innovation Award, a great opportunity to showcase our project nationally. We want to make our project known. In a second phase of the diffusion process, we hope to establish collaborations with other local, state, and national governments in scaling up our project. The potential for diffusion is considerable, as long as the call center technological solutions are available. We offer technical assistance to capacitate human capital to conduct scientifically valid telephone surveys of the population.

This innovation gained significant contributions from partners. To start with, we collaborate with partners to design questionnaires. The specialists on each specific theme are the governmental agencies we cooperate with. They are the ones who know which questions to ask. We assist in asking the questions in an unbiased format, avoiding distortions that may influence the responses we receive from citizens. Hence, we apply existing knowledge about questionnaire design, based on sociology and cognitive sciences, to improve the questions we ask. In doing so, we have learned tremendously about distinct issues in government, which is vital for the production of the final reports and recommendations we make on each specific topic. In addition, we work very closely with the subcontracted call center firm, especially in training interviewers and inputting questionnaires into their computational system.

An important contributor to this project is the subcontracted call center firm. They have been helpful in defining interview procedures, assisting in creating data entry interfaces, and pretesting questionnaires. Furthermore, the firm has assisted in cleaning the telephone number catalog by automatically verifying which numbers are valid and in use. This allows for increases in response rates. Furthermore, the firm hires highly experienced and capable telemarketing specialists trained specifically to conduct the interviews, improving data quality.

The partnership with and commitment of the private firm subcontracted to operate the government’s call center is fundamental for the project’s success. Finally, governmental agencies’ interest and courage in being evaluated and in understanding the efficiency of this modality of evaluation is what makes the project viable.

Our main product is the provision of qualified information about citizens’ perspectives on policy and service quality. Through information, we believe governments can improve their performance and better respond to society’s demands. The informational goods we provide are unique and exclusive in that they are not available from any other source. The public opinion data we collect in an expedite way is also scientifically valid and reliable. Finally, we obtain data with no additional costs to the government.

The main results of the innovation are the production of datasets and reports based on Computer-Assisted Telephone Interviews with citizens of the Federal District in Brasilia, Brazil. The focus is on popular perceptions and evaluations of the government’s performance in the provision of public services and the implementation of public policies. We produced 16 reports since the project was launched in 2015, assisting over 10 governmental agencies in improving the quality of their services and policy programs as well as the representativeness of their policy proposals.

These reports are available online for the Brazilian society to follow and learn about the performance of the Federal District’s Government. An additional feature of our initiative is its contribution to empowering citizens through knowledge of governmental performance. Our partners in the government have adapted, rescaled, and reevaluated their portfolio of services based on the public opinion data we provide. Furthermore, some use our findings to design new policy proposals and services. We hope to use the innovation in the future to continue unraveling the citizens’ views on new, emerging topics and trends, assisting the government to learn about complex social and economic issues through new, updated, and unique information.

The greatest challenge of the project is the construction of the telephone catalog with large coverage of the Federal District Territory. We went through several iterations using distinct registries of telephone numbers to compile the omnibus file we currently employ. The catalog in use is based on the list of possible telephone numbers according to rules provided by the National Telecommunication Agency (ANATEL) given local and area codes. In addition, we validate each number through automated dialing machines, constructing a final list of active numbers adding to over 4 million entries. In this way, we assure full coverage of the Federal District’s possible land and mobile lines. Therefore, the solution for the creation of the telephone catalog is one that can be implemented in all national territory, increasing the potential for scaling-up the project.

This initiative requires supporting infrastructure and services, in the form of a governmental call center, accompanied by contractual rules that allow for the conduction of active phone calls to citizens. It requires well-specified contracts that provide for sufficient numbers of attendants to conduct interviews and foresees the possibility of not just receiving phone calls, but also placing them. In addition, the presence of human resources specialized in social research, in particular, survey research, sampling, policy evaluation, and questionnaire design is necessary for the success of the project. However, technical assistance from Codeplan can easily qualify human resources to implement similar initiatives elsewhere. In fact, this is a goal of our diffusion strategy.

The problem the initiative confronts, the informational limitation of governments on policies and services evaluations, is a widespread public challenge. Governments everywhere want to understand how citizens evaluate the quality of public services and public policies as a means of improving their performance and the offer of outputs to society. Collecting public opinion data expediently, with low costs, and frequently is how our initiative innovates and advances upon existing alternatives.

The solution we propose is easily replicable once the required physical and legal infrastructure is available. However, it is common for governments in localities with significant populations and complex problems to hold call centers. Capacitating local human capital, another necessary step, is possible through the transfer of knowledge from the experience developed in Brasilia.

The major lesson from our experience is to use available resources – human, infrastructure and budget – to their fullest. In moments of scarcity, it is vital to do more with what is available or even with less. The Federal District today faces one of its most serious fiscal and economic crises, with severe cutbacks on governmental expenditures.

Efficiency - achieving desired results by optimizing costs – is key to the continuation of any policy. Our initiative moves the State in such direction, innovating by using technology to open a new channel of communication and information between State and society. A possible setback of the project is that hearing the people may cause discomfort to politicians and bureaucrats when evaluations are negative. It is important to bear in mind that the opposition and the media may use the information against the government. A solid communication strategy for data dissemination should accompany similar efforts elsewhere.

This innovation, like any other, is a consequence of the dedication, commitment, and vision of public servants concerned with increasing policy efficiency and improving the quality of public services. It is the fruit of individual motivation. Stirring innovation, however, is also a consequence of the environment one is in. The current Federal District Government is fully committed to innovation in the public sector: it is part of the government's deliverables to society. Therefore, this innovation, ultimately, is only possible because of the combination of these two factors.