OONI Measurement Aggregation Toolkit - Observatory of Public Sector Innovation

Governments around the world are censoring the Internet, but how can we document, track, and respond to Internet censorship events? In 2022, the Open Observatory of Network Interference (OONI) launched the Measurement Aggregation Toolkit (MAT) which hosts one of the largest open datasets on Internet censorship to date. In real-time, new measurements are published from around the world, enabling researchers, journalists, and human rights defenders to track and respond to censorship worldwide.

Innovation Summary

Innovation Overview

Government control over the Internet (in the form of censorship) often changes in correlation with political events. Social media platforms (like WhatsApp or Facebook) are commonly blocked during elections and protests around the world. Amid the Mahsa Amini protests, Iran tightened its grip on the Internet, blocking many more social media platforms. Following the war in Ukraine, Russia increased its censorship, blocking numerous independent news media websites. Following the military coup in February 2021, Myanmar implemented nightly Internet shutdowns for several months and blocked access to major social media platforms (which remain ongoing).

But most countries experience some form of Internet censorship. What differs from country to country is what is blocked, and the impact of the block. In countries where LGBTIQ rights are not recognized, access to LGBTIQ websites is blocked. In countries where minority rights are not adequately protected, access to minority group websites is blocked. In countries with low protections for human rights, human rights websites are blocked. But such cases of censorship can go unnoticed, or governments can seek plausible deniability (particularly since it can be hard to confirm a block).

How can we document Internet censorship around the world with data that can serve as evidence? Back in 2012, the Open Observatory of Network Interference (OONI) was created with the goal of enabling anyone around the world to independently investigate Internet censorship. To this end, they built a free software app (OONI Probe) that anyone can run to detect various forms of Internet censorship on their network. This app is run by hundreds of thousands of people in 241 countries and territories. Their censorship test results are published by OONI as open data in real-time to enable the public to investigate Internet censorship around the world.

In 2022, OONI launched the Measurement Aggregation Toolkit (MAT) which enables the public to track Internet censorship worldwide and to create their own custom charts based on real-time censorship measurement data. Specifically, the MAT can be useful for real-time tracking of new censorship events as they emerge (such as new blocks in Iran amid protests), as well as for longitudinal research (such as examining how censorship changes in a country over years, in correlation with events). The MAT can also be used to compare censorship across countries, and to examine the blocking of services worldwide.

Since the MAT hosts OONI data collected from 2012 to date, it is one of the largest open datasets on Internet censorship around the world. And every minute, as people around the world run OONI Probe tests, new data is published, offering new insight. Unlike other datasets on Internet censorship, the MAT is based on data collected from local networks, offering unique insight into how local Internet users experience Internet censorship.

Human rights defenders as part of the global #KeepItOn campaign regularly use OONI data to monitor and respond to Internet censorship events around the world. Journalists use OONI data to inform their reporting, and researchers explore novel research questions through OONI data. While OONI data was always publicly available, the MAT enhances such efforts in significant ways: researchers, journalists, and human rights defenders can easily examine censorship events with the click of a button (without performing data analysis)! The MAT incorporates OONI's data analysis techniques, enabling the public to easily investigate Internet censorship (and create charts) without the requirement of any technical skills.

In summary, the MAT enables the public to easily investigate Internet censorship worldwide based on real-time open data. As an outcome, we expect to see more decentralized rapid response efforts aimed at defending human rights on the Internet.

Innovation Description

What Makes Your Project Innovative?

OONI's Measurement Aggregation Toolkit is the only platform that enables the public to independently investigate and track Internet censorship worldwide based on real-time data. It is also the only platform of this scale that provides information about Internet censorship based on measurements collected from local networks, offering unique insight into how local Internet users experience Internet censorship. Moreover, the MAT is the only platform that enables the public to create charts based on aggregate views of real-time censorship measurement data.

What is the current status of your innovation?

OONI launched the Measurement Aggregation Toolkit (MAT) in April 2022. The platform has since been used extensively by researchers and human rights defenders in the Internet freedom community to track and respond to emergent censorship events. As an example, the MAT enabled human rights defenders to monitor censorship events in Iran amid protests, and in Armenia and Azerbaijan during border clashes.

Innovation Development

Collaborations & Partnerships

OONI has established partnerships with 41 human rights organizations, most of whom provided feedback over the years that informed the design and development of the Measurement Aggregation Toolkit (MAT). The MAT was specifically created to meet their needs, as they highlighted that they needed an easy way to view measurements in aggregate, track censorship events as they emerge, and have an easy way to visualize findings based on OONI data.

Users, Stakeholders & Beneficiaries

OONI partners are able to use the Measurement Aggregation Toolkit (MAT) to investigate Internet censorship as part of their research and advocacy efforts. Malaysia's Sinar Project, for example, uses the MAT to investigate Internet censorship across Southeast Asia. The global #KeepItOn campaign can use the MAT to inform their advocacy efforts against Internet shutdowns.

Innovation Reflections

Results, Outcomes & Impacts

Following its launch, the Measurement Aggregation Toolkit (MAT) enabled both OONI and other internet freedom community members to monitor and rapidly respond to emergent censorship events. As new censorship events emerged in Iran amid protests, OONI was able to rapidly respond with charts immediately generated through the MAT, based on real-time data collected from Iran. OONI subsequently published a new research report documenting the latest censorship events in Iran based on more extensive analysis, using MAT-generated charts. Quite similarly, OONI published a report on new blocks that emerged in Russia amid the war in Ukraine, using MAT-generated charts to communicate the censorship findings. Other Internet freedom community members also shared MAT charts on social media platforms, rapidly responding to censorship events.

Challenges and Failures

Creating the Measurement Aggregation Toolkit (MAT) presented several challenges (particularly for a small non-profit organization, like OONI):

Publishing measurements collected from around the world in real-time
Scaling the infrastructure to handle the ongoing expansion of measurement coverage
Ensuring the data quality of measurements
Determining how to visualize and present the data

To address these challenges, we migrated our infrastructure to new backend hosts, created an aggregation API, and performed extensive backend work to improve the availability and performance of our database and infrastructure. We also worked on improving the heuristics of our network measurement experiments and addressing issues to improve the data quality. We determined how to visualize and present information in the MAT through feedback collected from our global community of users.

Conditions for Success

Ultimately, our goal is to support a free and open Internet, where human rights (particularly freedom of speech and access to information) are protected. We measure the success of our work depending on how useful it is to human rights defenders and researchers in investigating and responding to Internet censorship. To this end, we track the number and types of research reports and advocacy efforts that make use of OONI data (which is enabled through the MAT). We also track cases where OONI data supports legal efforts that challenge the legality of censorship events. The success of OONI's Measurement Aggregation Toolkit (MAT) depends on:

OONI Probe users around the world (who contribute measurements)
Research and advocacy efforts that make use of the MAT
Community feedback that informs future improvement and development of the MAT

Replication

While OONI's Measurement Aggregation Toolkit (MAT) hasn't been replicated directly, there are a few complementary efforts (with whom we collaborate), and our community members have used OONI data to create their own, regional dashboards.

Lessons Learned

The single most important lesson that OONI has learned over the last 10 years is that it is essential to involve your community directly in the design and creation of your innovation. In order for an innovation to be useful and meet the needs of a community, it is important that they participate in its creation.

🇮🇹