The Open Data Portal guarantees that anyone, regardless of their technical knowledge, can move from doubts about Mexico City to answers based on data. Through an interactive data visualization tool, it seeks to transfer the power to analyze the information generated by the government to the citizenry. The portal is innovative because of its robust visualization tool and it having been fully developed in open source; allowing users to find data, visualize it, cross reference it and build stories, as well as upload their own data and interact with public data.
Innovation Summary
Innovation Overview
Like many cities, Mexico City has an Open Data strategy. To carry out their strategy, governments have two options: use open tools such as CKAN, which provides a platform for uploading data but does not have many features, or pay for expensive software that offers slightly more sophisticated services. We decided to create a solution that would incorporate the best of both worlds. The result: an open data portal built with open source that allows users to find data, visualize it, cross it and build stories, as well as upload their own data and interact with public data. Since January 2019 and continuously, the Digital Agency for Public Innovation, in coordination with the Public Administration Entities of Mexico City, has made the Open Data Portal available to the public where anyone can consult, use and download the databases generated by the government.
During this time, the Portal has undergone important changes, among which the migration from a licensed software platform (OpenDataSoft) to an open source platform (CKAN) in early 2021 stands out. Although this change meant savings of around $100,000 per year, it also meant the loss of several very useful features for citizens and for the government itself. Three years after the start of the current administration of Mexico City, and as those responsible for the implementation of the Open Data policy, it was necessary to reflect on the objectives achieved to rethink the strategy towards the future of Open Data in the City. Despite the great success of the platform, we realized that there was a huge opportunity for the data generated by the government to be used by an even greater number of people. In this sense, and under the principle of accessibility, in 2022, with financing from Bloomberg Philanthropies and the technical experience of DataSketch, we designed a strategy to reduce the opportunity cost of analyzing databases and eliminate information asymmetries.
The main component of this strategy is an interactive visualization tool that allows anyone accessing the Open Data Portal to build maps, treemaps, bar, line, and scatter charts with just a few clicks. The guiding principle of the project is that anyone, regardless of their technical knowledge in data, can move from questions about Mexico City to answers based on data. Like many open data portals around the world, the Mexico City open data portal is built on an open source platform (CKAN) and has file-level download functionalities in a non-proprietary format (CSV ), as well as a data API for querying through programming languages such as Javascript, Python, and R. These features, however, are useful almost exclusively for users with technical knowledge of data management and analysis. The new visualization tool, developed with R Shiny and connected to CKAN, seeks to compensate for these limitations and democratize access to information, providing an easy way to analyze and draw conclusions from data that is already open.
The new open data portal in Mexico City has a positive impact on regular users of the portal (academics, students or journalists) by facilitating the analysis of data generated by the government. However, the greatest impact is on the general public by allowing more and more people to consult and analyze the information generated by the institutions and public servants at their service. The main challenge was to design and develop a visualization tool generalizable to any data set. In other words, regardless of the structure, number of variables or file format, the tool is capable of generating a visualization that facilitates data analysis.
Innovation Description
What Makes Your Project Innovative?
The Mexico City Open Data Portal is innovative in two ways. First of all, in a technological sense, it is innovative because it is an open data portal developed entirely in open source software that has a robust and interactive visualization tool. It also represents a great advancement for the open data community because the visualization tool —built with Shiny— connects, through a new extension, to the most widely used open source platform (CKAN) in the world. Secondly, it is an innovative project because it revolutionizes the Open Data policy by prioritizing the use and analysis of data over publication. In this sense, we promote quality data that is accessible to more and more people.
Innovation Development
Collaborations & Partnerships
The success of the Portal is explained thanks to the collaboration of different allies. The participation of public administration entities to share databases in an open format has been essential. Likewise, citizen participation has been a pillar in the data publication strategy. For the development, the participation of Bloomberg Philanthropies and the Colombian company DataSketch was very important for the creation and implementation of the visualization tool.
Users, Stakeholders & Beneficiaries
Regular users (academia, journalists and civil society) are the first to benefit from the new portal by facilitating data analysis. However, the largest group of beneficiaries is the general public by providing an easy-to-use tool to explore the portal's data. Government agencies also benefit from the project because it contributes to improving public policy processes through evidence-based decision-making.
Innovation Reflections
Results, Outcomes & Impacts
The new open data portal in Mexico City is only a couple of months old, so it is not yet possible to talk about results, but rather about trends. One of the expected results is that more people explore the data from the Portal and new conversations are generated in the public sphere. Since the launch of the new portal, the average number of monthly users has increased by 36%, going from 50,000 to 68,000 users. It has also increased the average active time of interaction with the site from 45 seconds to one minute and 30 seconds, which is a good indication of the use of the visualization tool. In the medium and long term, it is expected that the average number of monthly users will remain constant at at least 65 thousand users and that the number of views generated with the tool and with data from the portal shared on social networks will increase.
Challenges and Failures
One of the biggest challenges we have faced is the process of thinking and designing the visualization tool. The challenge was to think of a tool with such general characteristics that it could be transversal to different data sets without sacrificing visualization functionalities. That is, to think of a tool that was capable of visualizing statistical bases and nominal bases. This entire process was resolved thanks to different work tables with the DataSketch team. The biggest failure we have faced is that we failed to launch the new portal on the desired date (July 2022) due to setbacks in installing the visualization tool on our servers. This was because Shiny is a new programming language and it required part of the team to fully understand how to build a Shiny development.
Conditions for Success
To achieve the success of the Open Data Portal, the following has been required: 1. A legal framework (Digital Operation and Innovation Law and the Data Management Policy) that empowers us to design and implement an Open Data and government openness strategy. 2. Have a team that is in charge of the operation and management of the portal as well as carrying out data quality control tasks. 3. Have a development team in charge of developing new features and bug fixes. 4. A project manager who leads the team and who is in charge of managing time and assigning tasks to each team member. 5. A couple of servers with the necessary capacity to host the open data portal and the visualization tool.
Replication
Three months after the launch of the new Open Data Portal, the tool has not been replicated, however, we have presented the project with some cities in the region (San José, Costa Rica; Rio de Janeiro, Brazil) and there is interest in replicating the project. One of the principles of the government of Mexico City is that all new development must be free software and must be available for donation. Specifically, this project has enormous potential to be implemented by other cities that, like Mexico City, use CKAN as an open source data management platform.
Lessons Learned
Promoting the democratization of information is an important accountability mechanism. In this sense, the idea that guided the entire strategy is that there is no more relevant data set than the one that is used. Under this same premise, we began the entire process of transformation of the open data plan and the Open Data Portal to focus more on the use of the data that we already have in the portal than on the publication of new data sets. Although it is important to have an open data strategy that focuses on the publication of information generated by the government, the truth is that we need to promote that the data can be used by all people.
Project Pitch
Status:
- Diffusing Lessons - using what was learnt to inform other projects and understanding how the innovation can be applied in other ways
Innovation provided by:
Media:
Date Published:
27 November 2023