This blog is the third in a series on a learning cycle of the UNDP Accelerator Lab in Mexico.  Together with the Performance Evaluation Unit of the Ministry of Finance and Public Credit (PEU-SHCP), we are looking for ways to enhance the capacity of the Performance Evaluation System (PES) to incorporate learning from those implementing public programs. In this blog we tell you how we came up with our first set of experiments, which involved the use of behavioral science to improve the interaction between the PEU-SHCP and the Responsible Units (RU) of budget programs. You can read the two previous entries here: first and second.

The Results Indicator Matrix

The PES is a system that involves multiple institutions and processes (learn more about it here ). Within PES, our attention was drawn to the process through which the RUs define, under certain guidelines and with the support of the SHCP, the indicators used to monitor their progress in achieving their goals. This process presents possible incentive games and adaptive behaviors that we found very interesting. By involving the implementers in the definition of the parameters with which their work is monitored, a proactive attitude is promoted that could mitigate the refusal to be evaluated, besides, who better than the implementer to outline what is possible to achieve given the design and resources of a program. On the other hand, it also opens the door to adaptive behaviors and the possible temptation to set conservative and easy-to-achieve metrics and/or goals.

The RUs are obliged to define the objectives of their programs, as well as the components and activities that they are going to carry out to achieve them, with the resources assigned to them. All of this must be captured in a Results Indicator Matrix (RIM) following the Logical Framework Methodology (LFM), where indicators that account for the results and impact of the programs are defined. Once defined, the RIM is uploaded to a platform and the RUs periodically report their progress.

What the MIR process looks like

This is the process on which we decided to concentrate the effort of our learning cycle. It begins when the PEU-SHCP notifies the RUs of their obligation and organizes training sessions so that the officials have the theoretical, methodological and technical skills necessary to design their RIM, load it into the system and follow up on their indicators.

After the training, the RUs make a first version of their RIM and upload it into a computer system managed by the PEU-SHCP, the Treasury Department Application Program (PASH in Spanish). The PEU-SHCP reviews the RIMs and, if deemed necessary, offers technical assistance for the RUs to improve the exercise. In this way, they refine the matrixes until they reach a final version.

Once approved, it is the responsibility of the RUs to collect the data to calculate the indicators and report progress in the PASH. The PEU-SHCP, which has access to all RIMs, and uses that information to conduct analyses and generate diagnoses and reports for decision-makers, as well as to maintain a platform where the public can consult information. The process lasts one year and each year programs can update their RIM indicators.

How we would like the process to go

We have an ideal, which we share with the PEU-SHCP, of how the RIM process should work. It starts with people who are senior enough to make decisions about program implementation taking an interest and attending training with their teams. Teams develop a solid understanding of the LFM and know the potential of Managing for Results (MFR) to improve the program and contribute to BRR.

Then, let them go back to their offices and integrate these tools as key parts of their work and not just as a requirement from the Treasury Department. Finally, all the information will be used by the PEU-SHCP to analyze, from a global perspective, the government's programmatic structure, and to generate diagnoses and reports of relevance for decision making.

Much of this happens, but there are parts where the system, as it is set up, encourages behavior that detours from this ideal. We want to understand the system and understand where these detours are generated in order to test alternatives to help us manage people's behavior using findings from Behavioral Science, the branch of knowledge that combines economics, psychology and neuroscience to better understand and explain human behavior and decision making.

The experiments

To familiarize ourselves with the process, we decided, together with the PEU-SHCP, on two entry points at which to design experiments. The first is the invitation that the PEU-SHCP sends to the RUs, and the second is the BRR training that the PEU-SCHP provides. In later stages of the cycle, we plan to explore other elements, inside and outside of this process, with which to experiment.

With the invitation, we want to try different ways of engaging with the RUs so that the invitation reaches those who make decisions about program design and the definition of performance indicators, and they become more interested in the PES. With the training, we want to try out different learning methods and changes in the educational dynamics in order to strengthen knowledge and make people more interested.

Are you a civil servant, do you know about the topic and/or have you received a BRR training from the SHCP? You can use this channel to share with us ideas on how to improve this experience and make people more interested in Managing for Results. At the Accelerator Lab we are creating the world's largest and most agile learning network on sustainable development challenges.