Partnering for success: a regional monitoring system for social equity and inclusive development
A multistakeholder partnership created a regional indicators-based monitoring system to track pro-poor health policy change across the Southern African Development Community (SADC), a low-income region beset by socio-economic costs of a high disease burden. This stimulated SADC’s Result-Based Regional Monitoring and Evaluation initiative. Extending to all SADC priority areas, Results-Based Monitoring and Evaluation (RBME) enables real-time tracking of regional performance, documentation of results at country level and the facilitation of evidence-based decision-making and learning
The innovation is a regional indicators-based monitoring and evaluation system for SADC. The need for a robust system was identified by SADC Secretariat in 2013/14, to extend its programme of work on poverty and ill-health and to help realise collective goals in the area of health in support of regional economic prosperity. The innovation responded to 3 drivers:
1. Significant socio-economic costs of the high disease burden in the SADC region plus common challenges to the region’s health systems and effective service delivery;
2. Understanding that international cooperation on a regional scale can enable collective responses to context-specific challenges beyond the scope of any one country to address unilaterally;
3. Evidenced support for region-wide social indicators-based monitoring.
The use of social indicators in regional monitoring captures ‘positive’ regional policies on health, social protection and human development that economic indicators miss. The SDGs focuses attention on the need for robust regional monitoring to track and evaluate resource use and the quality of change in ways that mobilise regional actors, institutions and partnerships.
The regional monitoring system aims to: support SADC countries and the SADC secretariat to identify achievements of and gaps in their socio-economic development programs; strengthen links between the regional body and member countries, facilitating integrated policy change; identify better mechanisms for data sharing and M&E of regional development programs; and enhance efforts to hold political actors to account for realising regional commitments to improve socio-economic outcomes.
The course of action was initiated through an international partnership between SADC Secretariat and The Open University (UK) in 2014-2015. A multistakeholder partnership forum involving 17 stakeholders from SADC secretariat, academia, civil society and government co-created the SADC-PRARI (Poverty Reduction and Regional Integration) Toolkit capable of measuring pro-poor health policy success and change. The strengths of the partnership, its working methods, and outcomes were seen in consensus within the partnership that: the major health issues prioritised within the SADC regional health agenda were those that most significantly affected those living in poverty; full implementation of extant SADC regional health policies had the potential to improve access to health services and medicines by disadvantaged majorities in the countries of the region; there was considerable unrealised scope to enhance the positive human development impacts of SADC regional initiatives, especially in relation to maternal and child health, effective health service policy implementation and health systems-strengthening; and there was a real potential to significantly strengthen regional capacity to improve health outcomes. The partnership favoured a social indicators-based monitoring system with input, process, output and outcome indicators (http://www.open.ac.uk/socialsciences/prari/).
In 2015 the SADC Secretariat initiated the Result-Based Regional Monitoring and Evaluation initiative (RBME). RBME scales up the essential features of the SADC-PRARI Toolkit, responding to the refreshed imperatives of SADC’s revised regional indicative strategic development plan (R-RISDP) (2016-2020). R-RISDP galvanised SADC member states redouble their efforts for a concerted region-wide strategy for integrated socio-economic development. RBME is directly related to the region’s results framework of the R-RISDP which is the main strategic plan for the SADC region: indicators are directly related to the R-RISDP and align with the annual operational plans coordinated at the Secretariat.
RBME was developed in line with the SADC-PRARI indicators-based planning and monitoring approach, and centrally concentrates on monitoring results at outputs, intermediary and final outcomes levels. The RBME initiative includes health and poverty, the focus of the SADC-PRARI Toolkit, but has wider scope and sector coverage. RBME's scope covers output-level monitoring of the R-RISDP, financial performance reporting, and regional instrument implementation monitoring. Its sectoral coverage includes poverty reduction and health, also extending to gender equality, social and human development, science, technology and innovation, industrial development and market integration; infrastructure support; and agriculture and food security.
RBME enables real-time tracking of performance, documentation of results at national level and evidence-based decision-making and learning, and thus greater accountability, nationally and regionally. The RBME is in its roll-out stage across the region's member states, and undergoes periodic audit and evaluation to identify challenges and responses to them. There is significant scope for scaling up and replication, aided by lesson-sharing on evidence-based policy making systems which is a focus of African Union architecture.
What Makes Your Project Innovative?
RBME is an innovation in regional M&E because:
1.It is region-wide and comprehensive in scope and coverage. The previous monitoring system was not inclusive of all SADC member states and/or not comprehensive in coverage of all major sectors or aspects of regional governance.
2. It has the capability to demonstrate the value and strength of intra-regional cooperation on social as well as economic affairs. Demonstrating the ‘value-added’ by regional social policy cooperation may stimulate greater regional investment and policy innovation at all levels to address shared socio-economic challenges
3. It introduces greater overall efficiency and effectiveness in regional monitoring. R-RISDP necessitated more robust and electronic M&E processes to realise regional goals (the previous system was based on manual inputs). This is a major step-change, enabling monitoring at Secretariat and Member State levels and greater accountability for resources and interventions.
What is the current status of your innovation?
RBME is being rolled out. Translation of the system into French and Portuguese is complete. By March 2019, 8 Member States will have completed their on-boarding. Extension to the other SADC member states builds on lessons learned from the past 2 years, focusing on enhancing the measuring of progress in achieving planned results at the output, outcome and impact levels and strengthening regional-national links to facilitate evidence-based decision-making and policy-making at national and regional levels. SADC Secretariat is undertaking an evaluation of RBME to strengthen its robustness and make it even more policy implementation-oriented. An audit/assessment is in the planning with a partner organisation and will focus on enhancing relevance, effectiveness and scalability of RBME. The module for monitoring SADC Protocols has been identified as a priority, starting with Protocols on Trade, Finance and Investment, and Human-social development of which health is a key component.
Collaborations & Partnerships
Partners in the 'pilot' Toolkit comprised officials in the health division of national ministries of health and/or social development; research institutes; international organisations, health-focused NGOs. Stakeholder engagement remains a key component of RBME roll-out, particularly the private sector and civil society. It is actualised through Strengthening National Regional Linkages programme which strengthens SADC National Committees by having non-state actors sit in key positions on the them.
Users, Stakeholders & Beneficiaries
Governments benefit from comparable and timely data available, enabling them to identify strengths and gaps in their development programmes across several sectors. Pilot evaluation identified officials' greater capability to use regionally-comparable data and significant capacity-building. Involvement of non-state actors in SADC National Committees strengthens regional-national linkages, enhances legitimacy in regional policy-making, and improves overall effectiveness of regional investment.
Results, Outcomes & Impacts
RBME has provided a framework and shared resource for measuring the realisation of the regional integration agenda's social policy goals through real-time performance monitoring. It has agreed indicators and data underpinning them. A measure of the results and impacts can be seen in the on-boarding of 8 of the 14 SADC member states, for many of which RBME marks a considerable step-change in M&E processes. Also, for example, SADC Council and Summit decisions are now regularly reported, based on action plans with concrete goals, targets and performance indicators for success. Full roll-out of RBME is expected with usage by all SADC countries across all SADC priority sectoral areas. Further evaluation of the value of RBME as perceived by users and stakeholders will need to be complemented by evaluation of how it enhances decision-making and policy-making in support of R-RISDP goals which must ultimately benefit citizens in raising social standards and inclusive development.
Challenges and Failures
RBME has generally worked well. A key challenge stems from a not so-robust consultative process during the development of the SADC RISDP Indicators at national level. A further challenge has been to integrate the roll-out and training of member states on the online system with actual programmatic issues with real data and in the context of the national structures. This has been complicated by changes to SADC National Committees and revisions to SADC stakeholder engagement policy. A further challenge has been to generate a jointly-agreed framework and timelines at national level on how reports and data are submitted.
Responses are largely addressed in a practical manner. Examples of responses include the SNRL programme to strengthen national ownership of RBME through the SADC National Committees, and a RBME regularly-updated user manual provided to member states as a resource/support measure. Some areas require more work, both in terms of process and technical aspects.
Conditions for Success
RBME requires the necessary policy environment, leadership, resources and infrastructural technologies to achieve desired results. A regional public policy tool such as RBME requires carefully-crafted rules of engagement and system security because of data sensitivity. Human, financial and technological resources are key to realising optimum attainment of both. RBME uses Role-Based Access Control (RBAC) and relies on member states following their existing internal communication procedures governing dissemination of information to other parties (in this case, SADC Secretariat). As RBME is electronic, it relies on reliable supporting infrastructure and services (notably, dedicated and reliable internet, ready and wide access to PCs/laptops). Leadership at policy and technical levels are key for success if commitment to updating the online system is to be realized. Cost-effective staffing and training strategies are essential, necessitating predictable and sustainable financing.
We are not aware of the innovation yet having been replicated in other regional organisations. In the future, we anticipate that RBME will be rolled out to the entire SADC region’s member states.
Beyond SADC, there are opportunities for replication in Africa and other low-income regions. SADC is one of the Regional Economic Communities (REC) that are part of the African Union Architecture where evidence-based policy measurement is emphasized. Many African RECs are developing concerted regional strategies for integrated socio-economic development, with enhanced social policy objectives as part of their economic development programmes, yet do not currently have robust results-based monitoring systems using positive (social) indicators. There is significant scope for learning from the SADC experience. Initially, the AU could potentially act as a forum to facilitate mutual learning, supported by South-South or Triangular cooperation mechanisms.
Regional-scale action is as an important mode of response to shared socio-economic challenges. Social indicators of positive regional integration need to be accorded greater weighting and priority to support comprehensive development planning that is genuinely socially-equitable and -inclusive.
Setting up a regional monitoring and evaluation system to realise this is a major challenge requiring political, technical and practical responses. Each country in the region has particular amalgams of states and publics, aspirations and limitations, circumstances and needed, external and internal pressures, and scaling this up regionally requires a bold vision sustained by robust leadership and diverse resources, nationally and regionally. Policy and technical leadership, robust government planning and data systems, policy frameworks and technologies for information-sharing on a regional scale are all essential elements of innovating on a regional scale and in ways that match the nature of the social policy issues to which the innovation is a response.
International partnerships are key to realising innovations in the public sector. Effective partnerships between governments, regional bodies, citizens and the private sector are essential to successful collaboration, ownership and implementation. There need to be sustainable funding mechanisms to support the systems in place at country level which will include commitment to up-skilling and placement of human resources to support these systems which should in line with the level of development of the regional economies in order to be sustainable.
Planning for the future requires measures in place to ensure system continuity and sustainability in the face of staff turnover and reallocation of duties. System roll-out must not be unnecessarily protracted as this adversely affects momentum. Functionality and usability need to be consistent and robust to instill confidence and provide seamless transitions.
This innovation report was prepared by Dr Mubita Luwabelwa (SADC Secretariat), Professor Nicola Yeates (The Open University) and Dr Themba Moeti (Health Systems Trust). This article does not necessarily reflect the views or opinions of the ESRC, DFID, the Health Systems Trust, the SADC Secretariat or Open University.
Dr Luwabelwa is Director of the Policy, Planning and Resource Mobilisation Directorate at SADC Secretariat. Professor Yeates is Principal Investigator (Director) of the ESRC-DfiD PRARI research project (Poverty Reduction and Regional Integration: www.open.ac.uk/socialsciences/prari/. ESRC-DFID grant reference ES/L005336/1). Dr Moeti is Director of the Health Systems Trust, South Africa. Dr Moeti and Dr Luwabelwa were members of the SADC-PRARI Toolkit partnership.