DOZORRO - Observatory of Public Sector Innovation

Innovation helps citizens extract value from open data on the use of public funds and fosters accountability of public authorities for goods, services and works purchased. With the help of the electronic platform, the citizens and business can complain about non-competitive behaviour, poor quality or check the efficiency of procurement by their city council, school, hospital. Introduction of AI-powered risk monitoring allowed anyone to find and act upon violations in a fast and simple manner.

Innovation Summary

Innovation Overview

Three years after the digitalisation of contracting, the e-procurement system (Prozorro) allowed to narrow the bottlenecks for corruption and saved about USD 2,5 billion for the national economy in 3 years. Rapid digital transformation and improvements introduced brought transparency and made public information on public contacting - operation worth 15% of the country GDP - accessible to anyone. However, the sector remained largely corrupted and given its volume - 4500 tenders per day - required close oversight to ensure compliance, equal access to the market and fair competition principles are being adhered. To give voice to those, whose rights are being violated, be it a taxpayer or discriminated business, Transparency International Ukraine launched DOZORRO - end-to-end feedback platform to accumulate and react to these violations.
The online platform allows business whose rights are violated to leave the structured feedback on the tender, buyer, another bidder etc. and the party to which the complaint is addressed to respond and eliminate the violation once confirmed. In case no reaction follows, the case could be forwarded to one of the experts to investigate the case. If the violation is validated, the appeal to the controlling bodies is submitted. The complainant has the opportunity also to rate the quality of responses from 1 to 5, and the tenders in which no response to the complain followed or where the satisfaction rate was below 3 are marked as risky and highlighted on the platform. Those tenders are prioritised for the review by civil society organisations monitoring procurement.
The innovation aims to unite advocates of the effective allocation of the taxpayers money to ensure equal rules in the public procurement and best value for goods, services and works purchased by the state for the benefit of the citizens.
We provide relevant expertise and deploy technical solutions enabling anyone - the supplier, the procuring entity, the controlling agency or any citizen - to identify, react to and report wrongdoings in public spending. TI Ukraine also seeks to channel the benefit of the open data on public spending to the civil society in the regions, including business and facilitate data-driven decision making in the sector of public procurement. The innovation has also proven to contribute to curbing corruption in the sector and shifted the anti-corruption paradigm in public procurement from the reactive mode to the preventive mode.
More than 930 000 of unique users have used the DOZORRO portal over 2 years and more than 84 000 of feedback units were recorded.
The system enabled to close the feedback loop and fostered trust by the business sector in the integrity of the public contracting process in the country. As all the feedback and track record of violations is aggregated on the portal in the supplier of buyer profile, anyone could review practice and integrity of the entity. The main beneficiaries are, therefore: a) procuring entities (local authorities, state enterprises) that are checking due diligence of the bidders to guarantee the best value for money b) businesses - to market research and the benchmark against other competitors c) investigative journalists conduction anti-corruption investigations d) citizens, able to check suppliers of the goods, services and works they are beneficiaries of (nutrition at schools, road reconstruction, public spaces)
e) oversight and prosecuting bodies - the bodies have opportunities to check for the violations in the public procurement to detect uncompetitive behaviour in a timely manner and react to them on the early stage.

DOZORRO is planning to deploy an electronic appeal mechanism that would direct the complaints to the respective controlling bodies for the review. To increase the usability of the innovation, the functionality that would allow receiving notifications for specific tender, bidder, buyer or status change the users subscribe to (for example, violations confirmed). In addition, DOZORRO uses open-source tools which make the system highly replicable elsewhere and already consulted Moldova and Poland on the application of similar functionality for their most prominent open data portals.

Innovation Description

What Makes Your Project Innovative?

The innovation allowed to convert transparency into the accountability in the sector of the public procurement and cultivate trust between civil society, business and state. DOZORRO made identification of the wrongdoings easier through the application of digital tools and open data processing which, in turn, narrowed down loopholes allowing for corrupted practices in public funds allocation. DOZORRO developed a public risk-based methodology based on 40 risk indicators part of which is devoted to the responsiveness of the procurement process parties to the complains submitted. Aggregation of all the feedback in one single place allowed to collect and manage all complain more efficiently, resolve them more quickly. DOZORRO also became the first to test machine learning for corruption risk detection in the domain of public procurement.

Innovation Development

Collaborations & Partnerships

DOZORRO is a showcase of “golden partnership” — with business willing to compete on equal footing, procuring entities ensure timely public service delivery and to greater value for taxpayers money on one side and civil society — striving to identify and react to cases of deviation from fair competition practices and public funds misuse — on the other. The innovation comes as a result of pulled efforts of TI Ukraine, grass root activists, tech talent, reform agents on the ministerial level.

Users, Stakeholders & Beneficiaries

Users of the portal are mostly local activists, investigative journalists, businesses already active or planning to compete for public tenders. Platform enabled a) local authorities, state enterprises to check due diligence of the bidders to guarantee the best value for money b) businesses to estimate trade-offs from joining different markets segments c) citizens and journalists see who supplies to schools, road reconstruction, public space and at which price c) policymakers analyse metadata.

Innovation Reflections

Results, Outcomes & Impacts

The innovation enabled participation in the process of control over public funds allocated for any internet users, with specific guidance and mechanisms on how to execute their rights to hold public institutions accountable. As a result of the DOZORRO work, one-third of the cases reviewed by the community of experts, are resolved either before the award of the tender - preventing violation - or afterwards ensuring liability of parties abusing competition. As of March 2019, more than 930 000 unique users have visited DOZORRO platform, with more than 84 000 feedback inputs recorded on the website.
Being a successful civic approach, the methodology spilled over to the government sector - DOZORRO advocated the use of the risk-based monitoring as obligatory for controlling body within the Law on Public Procurement. Recently, DOZORRO was listed in top 12 projects worldwide for engaging citizens as a corruption watchdog. Further advance of AI technology for risk flagging is planned in 2019.

Challenges and Failures

The biggest challenge in the deployment of the technology intense innovation was rooted in ensuring the usability of the portal for main beneficiaries and uniting the all-country community around monitoring activity. The deployment of the technology required constant knowledge update. Another challenge was linked with communication of the complex topic of the public contracting to the local activists engaged in handling the cases of violations submitted online at the beginning. The latter was addressed with the number of active communication channels online and offline with regular knowledge exchange sessions. In addition, TI Ukraine had an unwrapped network of activists well-familiar with procurement monitoring and loopholes. The usability of the portal is constantly being improved given capacity as a result of the feedback sessions with users.

Conditions for Success

Based on our experience we recognise collaboration, added value and timeliness of innovation among the crucial criteria. Engagement of the stakeholders coming from diverse background allowed to manage risks along the way timely and effectively. As well as unwrapping community of the activists ready to promote the use of the platform and disseminate the knowledge on how to use it was crucial for the innovation success. The launch of the automated system of risk indicators would be impossible without prior expertise that allowed to test the accuracy of the first pilot model.
In addition, focus on the added value for the users of the technology proved to be key and is the reputation as a most trustworthy resource for looking into when suspecting wrongdoing became the main asset allowing to interact with the beneficiaries and improve the service further. Technology is key, but the competent community enable to verify the quality of innovations is king.

Replication

DOZORRO shared its experience with leading Polish organisation working with open data - ePanstwo. We helped to deploy the functionality for feedback input on every tender on the https://rejestr.io/zamowienia_publiczne - platform that aggregates information on every procurement in Poland. The tool was presented on the Festival of the civic tech on April 6th in Gdansk. The project became a tangible outcome of the continuous networking and expertise exchange between the TI Ukraine and ePanstwo Foundation that resulted in transferring best practice of civic tech from Ukraine to Poland. The introduction of the feedback will allow integration of the procurement process stakeholders and citizens into the control over the public procurement online and is designed to improve the responsiveness of the contractors and trust in transparent procurement on behalf of the business.

Lessons Learned

Open data does not necessarily increase the level of trust to the public institutions. With the information-rich environment, it is easier to validate the concerns and events that trigger mistrust to the institutions and public officials. To channel the benefit of open data to the citizen an immense communication work should be carried out and on the ground, consultations have to take place. Transparency is not a remedy for a lack of trust.
In the case of Ukraine, the fact that technology originated in the civic sector instead of the governmental sector resulted in greater trust level and usability of the technology. Transparency International Ukraine as one of the main anti-corruption champions was a credible partner to unite all the stakeholders and ensure integrity of the launch process.
It is important to set high accountability standards also for those monitoring anti-corruption related activities. Since the 2017 TI Ukraine implemented the feedback management system for the organisations executing monitoring which means all the work is recorded and traceable in the system, fostering trust in the third sector among users.