Evaluating Public Sector Innovation: Support or hindrance to innovation? (Innovation Lifecycle 5a/6)

Overview

Governments around the world are facing unprecedented change and public sector needs innovation more than ever. Yet, how can we be sure that innovation in the public sector is actually delivering positive change? How can we be certain that innovations are moving in the right directions, that there is innovation happening in the public sector at all? Innovation is not inherently good or bad, but by definition, it is something new to the context it is applied in; and that brings a value shift. What this value shift is or who wins or loses, is not specified. With major changes with increased digitalisation on the way and the need to respond to complex policy problems, there is need for critical evaluation of the effects innovations are delivering to the public sector and if the sector itself is conducive to innovation in the first place.

At the same time, there is plenty of evidence that evaluations can themselves negatively affect innovation processes by ramping up risk aversion and fear of failure, sometimes locking in action to established practices. Evaluations may rationalise something that is often unintended, unplanned, creative activity, messy and uncertain by its nature. The novelty of innovation makes learning from the past and, thus, traditional ex post evaluations difficult if not useless, but evaluation may contribute directly to sustainability of innovation in the public sector itself.

This report outlines some of these core contradictions and conflicts between innovation and evaluation, which are probably the cause why no comprehensive public sector innovation evaluation framework currently exists. And maybe it should not exist. It might be more important to skill established evaluators on innovation principles and contradictions, rather than creating approaches for a specific community; and challenge the assumptions of evaluation methodologies themselves. As said, innovation is needed in all field in government and invariably if will be evaluated by various parties. Furthermore, innovation evaluation should be a continuous process (it cannot be otherwise as its aims and approaches are subject to change) and more reflexive practises may make more sense for innovators for learning then pre-defined criteria. Hence, the report argues that it is important to distinguish why evaluations are undertaken in the first place – for accountability, legitimacy, learning etc. Different aims may mean different approaches.

Furthermore, different types of innovation – in line with the OPSI innovation facets model – may necessitate diverging approaches as well. Thus, the report gives a broad overview of different clusters of evaluation (performance and impact evaluation, collaborative and user-centred approaches, in project reflection, automatized evaluations, future oriented evaluations, systems based approaches, systematic reviews and meta-analysis and triangulation and mixed methods) that have relevance to innovation due to their wide application or direct usefulness. The interactions between different methods and innovation require further analysis and empirical testing, and the report hopes to raise pertinent questions to accompany the former.