Digital Maturity Index for local governments - Observatory of Public Sector Innovation

Digital transformation is a long-term journey that involves complex changes. One of the challenges is how we evaluate the progress. We are inspired by the quote, "If you cannot measure it, you cannot improve it". Most of Catalonia's municipalities cannot do a digital assessment. The Digital Maturity Index provides an evaluation turnkey solution for local governments. It is based on open data, and it has a clear focus on public value impact.

Innovation Summary

Innovation Overview

The Open Government of Catalonia (AOC) is a public agency with the mission to foster the digital transformation of Catalonia's public sector, which includes more than 2,000 authorities: local, regional, universities, and agencies. It provides e-government services about shared digital platforms, interoperability, digital identity, analytics, and change management. AOC has collaborative governance, a citizen-centric strategy, and a turnkey approach to provide technology, support, data privacy assistance, legal advice, training, and best practices.

We are inspired by the famous quote, "If you can not measure it, you can not improve it," by Lord Kelvin. And according to OECD, "measurement is crucial for evidence-based policymaking; it helps to identify the need for policy intervention, enhances accountability, and improves the evaluation of the efficiency and effectiveness of policy actions."

Digital transformation is a long-term journey involving complex technology, processes, organization, and culture changes. One of the key challenges is how we evaluate the public value it generates. Most of Catalonia's municipalities are small and have many limitations in assessing their digitization process on their own. The Digital Maturity Index (DMI) for local governments provides a turnkey solution to evaluate their digital maturity. It is based on open data, and it has a clear focus on public value impact.

DMI is a composite of 30 key indicators that assess the degree of digital transformation of 988 local authorities: 947 municipalities and 41 counties. It is based on a theoretical framework of three main dimensions: digital rights, use, and openness.

Digital rights. Effective implementation of digital services and compliance with e-government regulations.
Use. The usage of digital services by the key users: citizens, companies, associations, and public employees. Therefore, we measure the digitization of the front office (G2C), back office (internal processes), and interoperability between public authorities (G2G).
Openness. The practical implementation of open government principles: transparency, right to access, open data, accountability, and participation.

DMI has become a powerful tool in Catalonia to build a data-driven culture for local governments. It provides a dashboard to assess the overall digital transformation in a territory to deliver insights for policymakers, analysts, and stakeholders. And a balanced scorecard the key indicator for a specific local government to support self-evaluation and better decisions of elected and government officials

The main objectives of the DMI are:

Create a culture based on data and evidence in local administrations.
Measure global digital transformation of territory to provide rigorous information to the personnel policy maker, analyst staff, and other stakeholders.
Evaluate key indicators specific to each local entity so it can know its status, thus facilitating decision-making when responsible for political and public personnel.
Identify strengths and areas for improvement of a local entity compared with similar administrations or those that lead the digital transformation.
Identify best practices of local administrations with excellent indicator results and detect those organizations that need additional support from a higher-level entity (county, province, or AOC).
Value the work of local entities leading the digital transformation in Catalonia.

The DMI is based on the following principles

Use of available open data.
Co-creation methodology with experts in public management.
Open consultation with local administrations to amend the available data or its methods.

There are several digital maturity indexes, but they are defined at the country level and may not be applicable at the regional or local level.

Electronic Government Development Index (EGDI)
Electronic Participation Index (EPI)
Local Online Services Index (LOSI)
Digital Government Index (DGI)
OECD eGovernment Benchmark
Information Economy and Society Index (DESI). European Commission
GovTech Maturity Index (GTMI).

However, they have been a reference to define the DMI.

Innovation Description

What Makes Your Project Innovative?

Developing an instrument to assess the degree of digital maturity of local governments based primarily on open data is a unique and innovative initiative, both nationally and internationally. Several digital government indexes are applied at the federal or state government level, which does not fit for conducting assessments at the local level. Moreover, most composite indexes are based on statistical data or surveys.

DMI is also innovative because it delivers a comprehensive set of visual dashboards using business intelligence tools that are addressed to the different types of users:

State / regional government officials: high-level dashboard and reports for a set of local governments depending on geography, size, and others.
Local Elected official: high-level dashboard with insights for a specific local government
Local government officials: detailed dashboard for advanced analysis

What is the current status of your innovation?

The Digital Maturity Index for local governments (DMI) is available for all governments of Catalonia. It is an essential tool for the regional government, the four provincial governments, the 41 counties, and the 947 municipalities.

AOC has created a data warehouse that integrates all the information from different open data sources using an ETL tool (extraction, transforming, and loading) to build the DMI.

The sustainability of this initiative is based on promoting that all the information required is available in open data formats based on standards and can be integrated easily into the data warehouse. Currently, 95% of the necessary information fulfills these requirements.

The primary data sources are:

AOC
Spanish Government
Provincial Governments
Autonomous University of Barcelona
Catalan Ombudsman
Public contractors: we exchange open data with contractors about some indicators (mainly back office) about 200 local governments.

Innovation Development

Collaborations & Partnerships

DMI is a collaborative initiative with the participation of different levels of government. It requires a high level of coordination, cooperation, and communication, with the participation of elected officials, government officials, and public experts.

The collaborators are involved in the following actions.

Stakeholder meetings
Workshops with public experts
Open consultation to review the draft results
Feedback platform to submit improvement ideas

Users, Stakeholders & Beneficiaries

Local government officials use DMI to know their situation, acknowledge their strengths, and define action plans. Private contractors are using DMI to offer specific services to the local government that provides poor services in some areas. DMI creates an evaluation and continuous improvement process. Citizens benefit from this culture. According to a benchmarking report we have conducted, citizens of Catalonia get much better digital services than citizens from other regions.

Innovation Reflections

Results, Outcomes & Impacts

The achievements are the following:

DMI is currently widely used by the Catalan government (state), provincial governments, county governments, and municipalities. The number of visits is 5.000 for the current edition.
DMI supports the evidence of improvements. Some of them are:
- The DMI has increased by 8 points since previous year.
- 76% of local governments have improved from previous year
- The usage of e-Services by citizens has multiplied by three, mainly because of the pandemic.
Satisfaction of using digital services has increased 2 points from previous years. We use the Net Promoter Score methodology adapted to the public sector.
AOC provides a consulting service to local governments to advise and support their digital transformation initiatives. Since the last years, the number of requests has increased by 10% to improve their DMI
We give an award to the top local governments divided into seven categories. This year, 76 local governments have been awarded.

Challenges and Failures

DMI is based on rigorous open data and that involves several difficulties:

There is no precise definition of the “digital government” concept and, therefore about the KPIs. There are different views and misunderstandings.
Structured and open data is not always available
Data does not always have a good quality.
There are semantic data issues. Although we use the same data labels, the meaning of the data is different depending on the source because the information is generated or collected used different criteria.

Because of all these issues, imply a huge work to obtain, standardize, validity quality, find alternatives sources and integrate all the data. To guarantee the quality of the results we conduct a public consultation releasing a draft report of the DMI. This process generates about 200 requests of changes or clarification and represents a significant effort.

Conditions for Success

The critical success factor of the DMI is to implement advanced data governance between different governments and stakeholders to gather all the relevant data in a standard, structured and quality gurantee way.

AOC has created a data warehouse that integrates all the information from different open data sources using an ETL tool (extraction transforming and loading) to build the DMI. The sustainability of this initiative is based on promoting that all the information required is available in open data formats based on standards and can be integrated easily into the data warehouse. Currently, 95% of the necessary information fulfills these requirements.

Replication

The methodology is replicable to local governments in any territory. We assess three dimensions, and we measure 30 key indicators. We believe the three dimensions are completely replicable: digital rights, use, and openness. And most of the 30 key performance indicators are also replicable with minor changes.

Most of the 30 key performance indicators are universal and can be replicable with minor changes. The most important are:

Digital rights

Website Search engine
Cartographic map
Urban planning map
eID
Business Permits
Accessibility
Cybersecurity
Interoperability
Online services
eGovernment regulation

Usage

eTransactions
eNotifications
Back-office digitization
Interoperability usage
Interoperability services
eTendering
Public contracts register

Open Government

Transparency
Open data
Participation
E-voting

Lessons Learned

Most of the DMI indicators are calculated automatically from the activity data of the digital services of the AOC Consortium and the open data of the electronic administration platforms (public and private). This change in focus notably reduces the data collection and validation procedure. Currently, the DMI is generated once a year, but thanks to the high efficiency and automation achieved, we plan to calculate it several times a year (at least for 90% of indicators).

The sustainability of this initiative is based on promoting that all the required information is available in open data formats based on standards so that they can be easily obtained and integrated. Currently, 95% of the information needed meets these requirements.
The DMI must be a collaborative initiative involving different levels of government. The four Provincial Councils of Catalonia and all the county councils must participate in the validation of the DMI methodology and results. Good governance

🇪🇸