Skip to content
An official website of the OECD. Find out more
Created by the Public Governance Directorate

This website was created by the OECD Observatory of Public Sector Innovation (OPSI), part of the OECD Public Governance Directorate (GOV).

How to validate authenticity

Validation that this is an official OECD website can be found on the Innovative Government page of the corporate OECD website.

PolicyKeys®: Where Can We Agree (AI Assisted Digital Engagement Game)

Screen Shot 2022-09-06 at 8.58.56 AM

We can agree on a lot more than we are led to believe. PolicyKeys is firstly a robust one-page narrative tool built around a ground truth so an AI can assist in predicting support for any public policy solution. PolicyKeys is secondarily a role-playing game to help player's better understand their own internal conflicts, as well as better understand their fellow citizen's differing beliefs—the result is a leaderboard of ideas that have deep and wide acceptance—approaching consensus.

Innovation Summary

Innovation Overview

The problem PolicyKeys solves is making sense of complex public policy puzzles, and to do so in an easy-to-understand fun weekly game. The one-page narrative tool helps the puzzle creator organize 32 Yes and No reasons, not cherry-picked, themed around the four-sides of a politically balanced table of 128 societal roles. The building of the puzzle gives many ah-ah moments for the organizers. The method allows users to score and rank every possible public policy solution. Citizens can play the sixteen games in a weekly puzzle to get a 360º view of the topic, puzzle, proposed solution, the best reasons to say Yes or No, which the roles might say and why, and at the end of the game be markedly better informed. Most players will come to realize we probably agree more than disagree, and the disagreements are more focused and more easily understood and discussed. Consensus ideas will naturally rise up the leaderboard, and perhaps become foregone conclusions.

 

PolicyKeys is designed to be a smoothie of crosswords (short clues), trivia (overall knowledge), role playing (empathy), mystery (solve the role’s motivation), poker (find each role’s tell), chess puzzles (best answers are so cool), jigsaw puzzles (how the roles fit together), and word games (sometimes there's no choice remaining but to guess). The initial AI is 256 distinct, seemingly deadlocked “supreme courts” of 9 roles, 128 with an implied 5-4 bias to change, and 128 with a 5-4 bias for status quo. After a puzzle is scored the AI identifies which remained deadlocked 5-4 and/or 4-5 which is an internal reliability error margin for the policy’s score.  Shockingly, while there is a meme that all public policy decisions are intractable, the gameboard/heatmap demonstrates that is not the case. We have scored over 60 solutions to public policy puzzles in the US and have a leaderboard of solutions (must have a majority from each of the four sides of the table) with median weighted averages of 2/3rds of the roles, and the best ideas in the 80th percentile.

 

Governments can use the tool themselves, partner with the private sector, and/or set-up a nonpartisan commission to narrate, rate, create and curate the best solutions. The private sector can do the same in the form of media outlets looking to compete to be the most trusted source for public policy leadership news.

Innovation Description

What Makes Your Project Innovative?

What's different about PolicyKeys is that it is role-based. People self-identify with some number of the 128 roles on the game-board. It becomes clear very quickly that these roles will have differing views of the proposed solution. The player comes to understand that it is natural to be conflicted about proposed solutions. This also presents as empathy for the roles in society that are not well known by the player. The ah-ah moment is that we agree on lots more than we are lead to believe by popular culture, and that everyone is the hero of their own journey and are acting accordingly. Nowhere in the world is there a leaderboard of solutions ranked by a truly nonpartisan score that can be be verified by replicability, mean reversion, and polling—until now.

What is the current status of your innovation?

We have launched a weekly public policy puzzle in the US in November. The game can be played for free at PolicyKeys.com, new puzzles on Mondays at 6 am EST. The book that the game is based on Politics 4.0: How Gamification, AI, and Idea Leaderboards Can Help You Depolarize the World is scheduled to be released Spring 2023 probably from Real Clear Politics publishing. There are 50+ weekly puzzles awaiting final quality checks before being published. For the future, the version one (V1) game is one-way in that the player does not communicate their choices to a database, while in V2 the communication will be two-way and the database will be a treasure trove of data to score likely public policy solution support.

Innovation Development

Collaborations & Partnerships

The innovation was developed indipendently but we are in talks to bring a node of the game to a US media outlet. We are aslo talking with democracy professors to use the tool in university. We envision that eventually a board will be needed to oversee the rules of the game, much like organized sports, so that the nonpartisan nature of the game won't be co-opted or have the appearance of being so. AI professionals will be needed to extend and innovate from our ground truth and initial pattern-recognition.

Users, Stakeholders & Beneficiaries

PolicyKeys will allow users to grasp the complexity of the public policy puzzle. Organizations can better craft their messaging about their advocacy. The answers (heat maps) are a roadmap for coalitions of the willing. Candidates might even use backing ideas on the leaderboard to set themselves apart from their opponent in competitive races.

Innovation Reflections

Results, Outcomes & Impacts

In the near future, we will be able to have data on the impact of PoliyKeys. Until this point, thanks to the game we could build the National Idea Leaderboard, which is made up of the solutions that have predicted support from each of the four sides of the politically balanced table. New and novel solutions are surfacing with surprising synergies of who should support it and why. For instance, we have two solutions for ending involuntary poverty, one is a tax credit for employers to pay a living wage before taxes, and another to raise the earned income tax credit for full-time workers to P50L (hallway between the poverty line and the living wage).

Challenges and Failures

The scoring of the puzzles is a challenge. We have had to fine tune the building of the puzzles, first draft (words), second draft (edit words and ascribe forecast reason numbers), third draft (AI scores for reliability), and a fourth draft which is a dress rehearsal edit of the game before going live. Calibrated individuals and team were very useful in catching forecasting errors, reason omissions, and typos. As we scale, we will have more eyes on the puzzles, and those eyes are likely to be more and more public-policy and subject-matter professionals. This will greatly add to the leaderboard's credibility.

Conditions for Success

Political Scientist talk about sample sizes in roughly these steps: 9, 30, 150, 400, 1000, 10K, 100K, 1M. Based on the gameboard, PolicyKeys should get to statistically significant sample sizes and reflectivity with society rather quickly. We'll need robust data science for V2, and probably partners who already know those markets, the science, and have followers.

 

By nature of the Topics, our puzzles are both prescriptive and descriptive meaning that sometimes the public is fairly well-versed in a topic and are trying to make up their minds (converge), and other times we will have found or created a new solution that is not well-known (diverge) and need to help disseminate this new solution. Therefore, the polling data or player scoring can be both descriptive (largely rear-view mirror) or prescriptive (largely focused on the horizon). It remains to be seen if there will need to be a staff of experts to prepare local puzzles or whether it can be crowdsourced.

Replication

Since PolicyKeys is a one-page narrative tool, standardized nonpartisan gameboard, an AI built-in for internal reliability, and is a classic presentation of the wisdom of crowds the results are highly replicable to the point of being able to be looked at through mean reversion and meta analysis. We are happy to train others how to do their own puzzles and we can help modify the gameboard to adapt to national differences.

Lessons Learned

It's been said that the best forecasters have models that are not too complicated nor too simplistic, and they make frequent and small adjustments. We have found this to be true. We may have started a puzzle a few months ago, and already the context of the puzzle has changed, or a law passed or trashed. The quality of the game is an important lesson as well. It is fine if a player thinks their answer was correct and ours was incorrect as long as they can see the logic in the game's call. It is not okay if the reason for the game's call was a typo, or based on an error, or obvious or nonobvious omission. Interesting differences in the calls of the different editors and different players is a teachable moment. It is here that a new and better signal might be found in the noise.

 

 

Anything Else?

If no one is curating a leaderboard of ideas explained in a trusted way, how does any citizen stand a chance of understanding these extraordinary complex times, and what and who to vote for? Having a standardized gameboard addresses this problem and allows for high scoring public policy ideas to more readily be shared and implemented by other countries.

Year: 2022
Level of Government: National/Federal government

Status:

  • Identifying or Discovering Problems or Opportunities - learning where and how an innovative response is needed
  • Generating Ideas or Designing Solutions - finding and filtering ideas to respond to the problem or opportunity
  • Developing Proposals - turning ideas into business cases that can be assessed and acted on
  • Implementation - making the innovation happen
  • Evaluation - understanding whether the innovative initiative has delivered what was needed
  • Diffusing Lessons - using what was learnt to inform other projects and understanding how the innovation can be applied in other ways

Innovation provided by:

Media:

Date Published:

15 November 2022

Join our community:

It only takes a few minutes to complete the form and share your project.