CONCERNS RAISED ABOUT THE PROCESS FOR DESIGNING THE KPI FRAMEWORK AND SETTING THE TARGETS

Author:

David Garmaise

Article Type:

Article Number: 2

After some delays, revised targets are expected to be presented to the Board in March

ABSTRACT The Board had been scheduled to approve proposed targets for the latest key performance indicators at its meeting in November 2016, but the decision was postponed because of concerns raised by Board members. Consultations are underway now on a revised set of targets, due to be presented to the Board in March. This article reports on concerns that have been raised about the process.

Since the Global Fund established key performance indicators (KPIs) and targets several years ago, they have been challenging for the Secretariat to generate, update and report on. They have also been challenging for the Board to oversee. Indeed, some Board members have admitted that they have trouble understanding the KPIs, many of which are very technical; the KPIs have been challenging for in-country stakeholders and other Global Fund watchers for the same reason.

We have been writing about the KPIs in GFO since 2004. The KPIs have always seemed to be in a constant state of evolution. For the 2017-2022 Strategy, the Global Fund decided to make some major changes to the KPIs. On 15 June 2016, the Fund adopted a new framework with three tiers of indicators illustrated by a pyramid. At the top of the pyramid are the strategic KPIs; in the middle, implementation KPIs; and at the bottom, regular thematic reporting (see GFO article.)

Most of the attention so far has been on the strategic KPIs. In June 2016, the Board also adopted 12 strategic indicators divided into five categories:

• KPIs tracking the Fund’s mission-level strategic targets;
• KPIs tracking Strategic Objective 1: Maximize impact against HIV, TB, and malaria;
• KPIs tracking Strategic Objective 2: Build resilient and sustainable systems for health;
• KPIs tracking Strategic Objective 3: Promote and protect human rights and gender equality; and
• KPIs tracking Strategic Objective 4: Mobilize increased resources.

(See GFO article describing the 12 indicators.)

The Board was expected to adopt targets for the strategic KPIs at its Board meeting in November in Montreux, Switzerland. The proposed targets had been discussed and proposed to the Board by the Strategy Committee (SC) and the Audit and Finance Committee (AFC). However, Board members were not prepared to adopt the targets. Many felt that more work needed to be done. As reported in GFO, the Board requested that members submit statements, questions, concerns, or suggested revisions regarding the performance targets; and that the Secretariat provide a response. The Board also asked the SC and the AFC to establish a joint Advisory Group to work with the Secretariat to present revised targets. Representatives of technical partners were to be included on the Advisory Group. The plan is that the revised targets will be presented to the Board by the first week of March 2017.

Board members have provided their input on the proposed targets; the Secretariat has responded; and the Advisory Group has been formed. Consultations are planned for January and February 2017.

Since this matter is currently under active consideration by the Board, GFO is not able to report on the proposed targets that were presented in November, or on revisions or additions to these targets that are now being discussed by the Advisory Group. However, we can report in general terms on concerns that have been raised about the process and about the KPI framework.

In a document which it sent to the chairs of the Strategy Committee and the Audit and Finance Committee, the leaders of the Board’s Implementers Group expressed concerns that their member countries have not had a leading role in the process of target-setting to date. The Group expressed other concerns which can be summarized as follows:

• The targets have to be realistic. If they are not, we risk not achieving the objectives of the Strategy.
• The methods used to establish the targets are very technical. They use mathematical modeling, data, and assumptions that are not readily accessible to most people.
• Some of the targets are based on models that are, in turn, based on estimates.
• The Secretariat and its partners should explain how the targets are derived. Without this information, it will be difficult for countries to come up with national-level targets.
• The process of developing the targets has been too rapid.
• International experts have led the process, while governments, civil society, and affected communities have had little input.
• The models may align with the global plans of the three diseases, but they don’t take into account realities on the ground – such as significant gaps in data, weak health systems, absorption issues, human rights barriers, armed conflicts, and natural disasters.
• The modelling assumes significant increases in domestic financing. However, recent economic forecasts warn of increasing uncertainty in the global economy.

The implementing countries would like to see practical conversations on target setting in implementing countries. In the future, they would also like to see targets established and aggregated from the ground up.

Finally, the implementing countries would like to see the consultations that the Advisory Group is organizing in January and February be done in person rather than by teleconference, to ensure the best results.

The full text of the Implementers Group document is available here.

Other feedback

Aidspan invited all Board constituencies to comment on the KPIs in general and on the process for establishing the targets. Two constituencies replied: Point Seven, and a constituency that wishes to remain anonymous.

Ambassador Carsten Staur, who is the Board member for the Point Seven constituency and also Permanent Representative of Denmark to the U.N. in Geneva, told Aidspan that the constituency preferred not to discuss the process at this time. However, he provided the following general comment on KPIs:

“We attach great value to the KPI framework as an important tool to monitor the progress and results of the Fund and, thus, the achievements of our common goals. Although there are challenges in creating a comprehensive and relevant framework that satisfies all concerns, we are hopeful that we will be able to adopt the appropriate targets for the Fund shortly. In this regard, we are pleased with the Funds increased efforts to improve data systems, to ensure enhanced data quality. In the end, the KPIs will only be as efficient as the data available to report on them.”

The other constituency raised a number of concerns about the process. A representative of the constituency told Aidspan that it is obvious that the Board was not satisfied with the process for setting the KPI framework in the spring of 2016 and the process for setting the targets in the fall of the same year. The representative pointed to the fact that each time proposals from the Secretariat were not approved during the Board meetings.

With respect to the process for setting the KPI framework in the spring of 2016, the representative said that the process that was followed after the Board meeting did not constitute an inclusive dialogue. The constituency had expected a meeting to be organized to discuss the framework and reach an agreement. Instead, the Secretariat responded to comments and an electronic vote was held, resulting in a slightly revised framework.

For each round of consultations, the representative said, the Secretariat is trying to organize preparatory meetings with some constituencies (such as members of the committees) and with technical partners, but the timing of the meetings is often very tight; exactly who gets to participate in these meetings is often not clear; and the comments from participants are not necessarily taken into account.

Finally, the representative said, it would be helpful if, in future, a stronger, more inclusive process – known to all ahead of time – could be designed.

Further feedback?

Other Board constituencies that would like to comment on this article’s topic, should contact the author (david.garmaise@aidspan.org). We will endeavor to include such input in an upcoming issue of GFO.