Prospective Country Evaluation Extension Synthesis Report

Author:

Aidspan

Article Type:

Article Number: 4

The Global Fund's Technical Evaluation Reference Group reports on the extension’s findings

ABSTRACT The Technical Evaluation Reference Group conducted Prospective Country Evaluations in eight countries for the allocation period 2017- 2019, followed by one additional year and a three-month extension. In October, a position paper was presented to inform the Strategy Committee of the results for the three-month extension period and the TERG’s recommendations, and to request submission of the document to the Board for information.

Background

The Board approved the Technical Evaluation Reference Group (TERG) to commission Prospective Country Evaluations (PCE) in eight countries for the allocation period 2017- 2019, followed by one additional year and a three-month extension. In October, a position paper was presented to inform the Strategy Committee (SC) of the results of the PCE Synthesis Report for the three-month extension period and the TERG’s recommendations, and to request submission of the document to the Board for information.

PCEs are in-depth, country-level, prospective evaluations that utilize a variety of methods to provide a detailed picture of the implementation, effectiveness, and impact of Global Fund grants in selected countries. The goal of the PCEs is to independently assess ongoing program implementation and impact at the country level to generate evidence and inform global, regional, and in-country stakeholders in order to accelerate the progress towards achievement of the Global Fund Strategic Objectives (SOs).

From April to June 2021, the PCE conducted some final analysis in relation to grant revisions, resilient and sustainable systems for health (RSSH) investments, and grant-making. Key areas explored during the extension phase included:

1. New Funding Model 2 (NFM2, allocation period 2017-2019) grant revision issues and any relevant lessons learned from the Global Fund’s response to COVID-19 and particularly operational flexibilities introduced in 2020;
2. The understanding and use by Country Coordinating Mechanism (CCM), government and other country stakeholders of the terms health systems support and health systems strengthening;
3. Reasons for the limited uptake of RSSH coverage indicators in NFM3 (allocation period 2020-2022) grant performance frameworks; and
4. NFM3 grant-making, including drivers of budgetary shifts for RSSH and equity-related investment and transparency, country ownership, and inclusion.

This article outlines the findings and recommendations from the three-month extension of the PCE, and the TERG’s position regarding these findings in a report presented to the 17^th SC meeting in October for its information and consideration. It is based on the Global Fund Prospective Country Evaluation Annex To 2021 Synthesis Report Findings and recommendations from extension period dated 30 June 2021 and the Strategy Committee document GF/SC17/04A, Technical Evaluation Reference Group: Prospective Country Evaluation Extension Synthesis Report.

Key findings

Eleven key findings contained in the PCE Extension Synthesis Report were presented in relation to grant revisions, RSSH investments, RSSH coverage indicators, and grant-making:

NFM2 Grant Revision issues and lesson learned from COVID-19 response:

1. NFM2 grant cycle revision processes were burdensome due to both the length of time for decisions to be made and the large number of participants or layers of decision-makers.
2. Making updates to the performance frameworks was not raised as a factor contributing to grant revisions being burdensome.
3. Flexibilities introduced in response to COVID-19 have increased the speed and efficiency of grant revision processes in the final year of the NFM2 grants.

 

RSSH Investments (the understanding and use of health systems support and health systems strengthening (HSS) by country stakeholders):

4. For RSSH investments, a set of business models and contextual factors explained the predominance of health system support over strengthening investments in final grants, including:

• The three-year grant cycle;
• Lack of guidance in allocation letters on how much to invest in RSSH and what types of investments to prioritize to strengthen the health system;
• Lack of planning around RSSH at the grant design stage;
• Overall health system resource constraints which lead to RSSH funds being used to fill gaps; and
• Lack of participation of health systems experts, including from other donors/partners engaged in this space, in funding request design.

5. Grant RSSH investments varied in their alignment to national health system objectives and their harmonization with other external partner investments. There appeared to be limited landscape and/or gap analysis to guide how and where to focus Global Fund investments alongside domestic and other donor efforts. Nonetheless, there are examples where investments added value to those of others.

 

Reasons for the limited uptake of RSSH coverage indicators in NFM3:

6. Most grants that included RSSH modules had relatively few RSSH indicators in performance frameworks. Several business models and contextual factors influenced this, including:

• RSSH performance indicators are not mandatory;
• Concerns about being held accountable for poor performance on RSSH indicators;
• Data to assess performance are not readily available and/or expensive to collect; and
• RSSH performance indicators are poorly aligned to investment areas.

 

NFM3 Grant Making including drivers of budgetary shifts for RSSH and equity-related investment:

7. In five countries, grant allocations to RSSH increased during grant-making; while they decreased in two.
8. Due to omissions in the breakdown of budgets by key population (a new tab in the detailed budgets), it was not possible to assess shifts in human rights, gender, and equity during grant-making.
9. Reasons for changes to budget allocations included: reclassification or changes in budget approach; changes in implementation arrangements; Technical Review Panel (TRP) recommendations; and concerns around achieving grant performance (absorption) targets.
10. Overall, countries reported country ownership to be higher during NFM3 grant making, compared to NFM2, although this related more to government than civil society stakeholders.
11. Despite greater country ownership, final grant budgeting processes remained opaque in most countries and final financial decisions were often made separately from technical discussions. Principal Recipients (PRs) did not always formally communicate changes to implementing partners.

 

Report Recommendations and TERG Position

The TERG endorsed these eleven key findings and, based on these, the new recommendations from the report are:

Recommendation 1 (revised to integrate findings from the 2021 extension phase): In order to reduce gaps between policy guidance and grant design, improve communication around how to invest more strategically in RSSH, including community system strengthening (CSS).

• In the next Strategy, the Global Fund should clarify its position on whether the primary objective of investments in RSSH is intrinsically tied to the management of the three disease epidemics; or whether its goal is more broadly to contribute towards health systems strengthening.
• Within the current Strategy and as the Global Fund moves forward, the Secretariat should improve the consistency of its communication on what these objectives are and how to invest more strategically (and less as a gap filler) in RSSH.
• Continue to embed the RSSH Roadmap and build on current guidance notes by working with individual countries to clarify specific Global Fund RSSH priority areas and what health system strengthening (HSS) as opposed to supportive investment would look like for these. This should include ensuring that grant activities have a specific short- and long-term purpose, ownership and accountability structures, and with indicators and targets in performance frameworks that relate to both implementation and intermediate outcomes as well as longer-term outcomes which may span over several grant cycles
• To facilitate alignment of Global Fund RSSH investments to national health strategies, ensure proper engagement and ownership from health system planning experts and leaders in National Strategic Plan and Funding Request development processes, and implementation oversight.
• To aid coordination and harmonization with other partners: (a) work with technical partners to ensure agreed tools/processes for HSS landscaping (e.g., gap analysis) are in place; and (b) support countries to operationalize these tools to guide investment prioritization with both national health priorities and other partner investments.

 

Recommendation 2: In order to improve grant contribution to equity and SO3, explicitly promote grant investments in these areas, including through more direct measurement of the drivers of inequity and of outcomes of human rights and gender investments.

• Invest more in data and data use, including up-to-date key and vulnerable population (KVP) surveys as well as other data sources that shed light on socio-economic, gender, geographical, and ethnic differences in disease burden and access to services that grants are aiming to contribute to.
• Ensure performance frameworks incorporate existing data on human rights and political commitment as well as disease burden and service access amongst different population groups and use this data effectively to monitor grant contribution to both SO3 and SO1 impact.
• Recognizing the success of strategic initiatives and/or matching funds in incentivizing grant investments in reducing equity, human rights, and gender-related barriers to accessing services, prioritizing scaling up across the portfolio, and incentivizing such investments through mainstream grant management operations. This should include explicit efforts to improve implementation and where necessary, timely revisions to maximize grant contribution to reducing barriers to care and disease impact.​

 

Recommendation 3 (based on evidence gathered during 2021 extension phase): Ensure that in future grant design processes, grant-making is as transparent and inclusive as the funding request process, and that greater efforts are made to maximize country ownership of the final grant awards, including participation by a wider group of stakeholders.

• Inclusivity and participation:
• Clarify the final grant budgeting process to all stakeholders, including information on who should be involved, what is expected of them, and training where required to improve participation.
• Where possible, select Sub-Recipients (SRs) earlier in the process to facilitate their participation.

• Transparency and country ownership:

• To Improve transparency by systematically documenting, and sharing with stakeholders, the significant budgetary changes to grant design and implementation, including the rationale and technical consequences of financial changes.
• Build ownership beyond the national government/Ministry of Health by engaging in more consultation with SRs/civil society organizations (CSOs) whose activities are changed as a result of grant award budget changes.

 

Recommendation 4 (including further information from COVID-19 operational flexibilities lessons during the 2021 extension phase): Build in more flexibility and responsiveness in implementation by simplifying grant revision processes to encourage their use throughout the grant cycle.

• Consider flexibilities and streamlining of material program revision process to encourage/reward earlier introduction of innovative programming that maximizes impact and limits non-strategic budgetary shifts to later in the three-year grant cycle.
• Introduce flexibilities to PR and SR contractual arrangements and performance frameworks that can be used to introduce mid-term changes as required.
• Through the Secretariat’s planned grant revision review (mid-2021), examine how countries could strengthen data-driven revision decisions (thereby avoiding the over-reliance on financial data to guide revision decisions), in line with establishing a more streamlined, flexible process for program revision.
• Building on lessons learned from the introduction of COVID-19 exceptional operational flexibilities introduced to minimize the impact of the COVID-19 pandemic on supported programs and Secretariat operations, adapt current periodic budget review processes such that adjustments to scope/scale can be included (akin to a program revision) but with rapid response/turnaround times and a streamlined revision process to reduce the level of burden imposed on stakeholders, such as by:

• Clarifying what constitutes a scope/scale revision and relaxing the requirement for TRP engagement for minor changes;
• Reducing the level of information required from PRs, and possibly the Secretariat, through the grant revisions request forms and wider process;
• Maintaining more flexible and/or electronic PR and CCM endorsement processes and reconsidering the number of different entities required to approve/endorse proposed revisions.

 

Recommendation 5 (revised to integrate findings from the 2021 extension phase): Improve grant-specific performance monitoring to inform implementation decisions.

• Establish routine grant review processes at the country level with a quality improvement lens, emphasizing grant-specific performance data and drawing on emerging evidence and data to better inform revisions that maximize impact.
• Implement proposed reforms of the grant rating system to reflect both grant-specific performance and contribution of grants to national program performance. Additionally, this should draw upon qualitative inputs, including expertise of the CCM, Local Fund Agent, Country Team, and wider Secretariat.
• Based on the revised grant rating system, develop a set of indicative options to demonstrate how good and poor performance could be responded to, and a framework for deciding when and how to introduce these measures in different contexts and circumstances.
• In relation to RSSH specifically:

• Strengthen use of revised RSSH indicators to address delayed implementation and potential deprioritization throughout grant implementation.
• Support country stakeholders to ensure that appropriate indicators are included in Performance Frameworks to monitor RSSH investments and progress towards health systems strengthening, at least for larger RSSH investments in the portfolio.
• Ensure that indicators add value to national efforts to track progress towards RSSH as well as support Global Fund efforts to measure RSSH outcomes. Where appropriate, they should harmonize with other global M&E frameworks for HSS, primary health care, and pandemic preparedness.
• Ensure performance incentives posed by the business model do not deter investment in health systems strengthening. Potential actions could include:

• Set grant targets over successive grant cycles, perhaps with interim targets for each grant cycle (this would need to be informed by a longer-term RSSH investment strategy).
• Measure RSSH implementation progress that directly relates to PR actions, ensuring other engaged stakeholders, including Ministries of Health and Finance, are held accountable for implementation progress on activities for which they are responsible.
• Enable more flexible use of grant funds and/or make concessions on reporting of absorption, such that poor rates of absorption for RSSH do not reflect poorly on PR performance.

 

Conclusion

The TERG in large part endorsed the revised recommendations, which are adjusted based on the findings from the extension period.

Proposed next steps

The SC and the Secretariat were requested to consider the above findings and considerations for improving the Global Fund model as well as clarifying the next Strategy. Additional details are available in the PCE Extension Synthesis Report.

Concurrently, the SC was requested to consider the essential role country-based evaluations are playing to generate evidence and inform global, regional, and in-country stakeholders in order to accelerate the progress towards the SOs of the current and next Global Fund strategies. The TERG and the Global Fund Secretariat published the extension period report as an annex to the PCE 2021 Synthesis Report, along with a TERG Position Paper and a Secretariat Management Response.