Audit of the COVID-19 Response Mechanism 2021

Author:

Aidspan

Article Type:

Article Number: 5

This worthy mechanism has still to be fashioned to respond effectively to emergency crises

ABSTRACT This article summarises the Office of the Inspector General audit report of the COVID-19 Response Mechanism. It is a long article because the audit report is lengthy and contains a lot of detail, hardly surprising since the amount of money involved, more than $ 4 billion, was considerable and the operation of the mechanism is intrinsically complex. There is a balance of achievements and issues but, given the many findings, the Agreed Management Actions are few. That said, by the time those actions are completed according to the agreed timescale, they may not materially improve the outcomes of the mechanism.

In 2020, in response to the COVID-19 pandemic, the Global Fund developed a new funding modality, the COVID-19 Response Mechanism (C19RM) with the objectives of funding interventions at country level that focused on:

1. COVID-19 control and containment interventions (e.g., personal protective equipment (PPE), diagnostics, treatment, and communication – as per World Health Organization (WHO) guidance).
2. COVID-19-related risk mitigation measures for programs to fight the three diseases (HIV, tuberculosis and malaria).
3. Expanded reinforcement of key aspects of health systems, such as laboratory networks, supply chains, and community-led systems.

In tackling these objectives, many different activities and interventions were funded. Overall, the mechanism focused on investments in commodities at country level, with a strong reliance on the Global Fund’s existing procurement mechanisms.

The success of this C19RM version 1.0 led to its extension in 2021 to version 2.0. While created with the same overall goal in mind – supporting countries in their fight against COVID-19 and to preserve HIV/AIDS, TB and malaria gains – there are key differences in the nature and context of C19RM 1.0 and C19RM 2.0.

C19RM 1.0 was designed and rolled out in the immediate aftermath of the announcement of the pandemic. It considered the Global Fund’s past experience/existing guidelines related to emergency funding and was adapted for speed, with a clear focus on a rapid response. In comparison, C19RM 2.0 was a much more significant investment, with a longer implementation timeframe. It was designed in part through leveraging lessons learnt from its predecessor.

Given that C19RM 2.0 investments were four times greater than C19RM 1.0, the Board emphasized the need to balance speed with robustness, to match the level of investment, risk, and ambition.

The key changes were:

• A longer time horizon in which requests could be approved, with implementation to be completed by end of December 2023.
• The establishment of a more comprehensive allocation model (including qualitative adjustment considerations).
• A tiered review process to support urgent needs for COVID-19 health products (Fast Track) and more comprehensive requests for additional interventions (Full Funding and Supplementary Funding).
• Creation of more processes and support mechanisms to increase Country Coordinating Mechanism (CCM), civil society organization (CSO) and key population engagement, to require engagement with the national COVID-19 response coordinator, and to align National Strategic Preparedness and Response Plans for COVID-19.
• An additional technical review process with key partners (known as GAC-CTAG).

The Office of the Inspector General (OIG) audit report GF-OIG-21-008 published on 15 April can be downloaded here.

Key considerations Impacting C19RM 2.0

C19RM 2.0 was designed considering a number of inherent tensions in terms of how it would be implemented and how it would be monitored.

Design decisions were often made under pressure and with incomplete information. In responding to countries’ urgent needs, the Global Fund had to balance considerations of speed, robustness, equity and inclusiveness in its investment decisions.  There is no absolute “correct” balance between these considerations.

Here are some examples provided by OIG of trade-offs required in deploying significant resources at pace:

A total of $3.54 billion was made available for C19RM 2021.  Of this, the Investment Committee approved amounts totalling $3.194 billion, the split across the key objectives being as follows:

Of the approved amounts $2.12 billion was spent on health products.  The biggest health cost categories were: oxygen-related costs (24%), PPE (23%) and diagnostic products (21%).

Audit objectives, scope and ratings

This audit assessed the adequacy of C19RM in providing support to countries through C19RM funding approved by the Global Fund Board.  The audit assessed:

1. The design and effectiveness of the C19RM 2021 approval process and related reporting to the Global Fund Board; this includes the governance, processes, systems, tools, and people in place to ensure that funding requests are processed in a robust manner, that funds are made available at country level and that the Global Fund Board is continuously informed of progress.
2. The design, and where possible implementation, of monitoring and oversight mechanisms to ensure effective and proper utilization of additional funds at the country level.
3. The timely utilization of C19RM 2020 funds made available at country level, where possible; this includes identifying challenges and root causes preventing the effective utilization of these funds.

The audit focused on the design and effectiveness of components that support the forward-looking implementation phase.  Where possible, the audit assessed C19RM 2020 funds utilization and in-country challenges by leveraging OIG country audits.

The audit scope excluded the following processes:

1. C19RM 2021 allocation methodology, except for qualitative adjustments made as part of the funding request review and approval.
2. The recording of contingent liabilities.
3. The in-country supply chains processes that relate to C19RM investments.

It was not possible for OIG to fully review the implementation and utilization of C19RM 2021, or the effectiveness of the Monitoring & Oversight (M&O) framework whose implementation was not complete at the time of the audit.

CR19RM grant request and approval processes were examined across ten countries: Guinea, India, Kenya, Malawi, Mozambique, Nigeria, Philippines, South Africa, Tanzania, and Ukraine.

The audit ratings of objectives were as follows:

Key Achievements and Good Practice

1. C19RM 2.0 maintained a rapid pace in approving requests: Within three months of its launch, it had awarded more funds than the entire 2020 allocation. The approval rates of both C19RM 1.0 and 2.0 were similar, a significant achievement given C19RM 2.0 had to process and approve four times the $ amount of funding requests than C19RM 1.0 – see the following graph:

Award Rates of C19RM 1.0 vs C19RM 2.0 (as of December 2021)

2. Improved inclusiveness: The Global Fund Secretariat made significant efforts to enhance the inclusiveness of the C19RM 2.0 process through: (i) increasing CCM budgets in 2021 to support improved engagement of CSO and key and vulnerable populations (KVPs); (ii) improvements to application material and guidelines, including mandatory lists of communities’ priorities in funding requests (FRs); and (iii) increasing funding from Centrally Managed Limited Investment funds, which were used to strengthen CSO and KVP engagement.
3. Well-designed M&O framework: There was a clear positive evolution in 2021 compared to 2020.  For C19RM 2.0, the Secretariat developed a broad framework for M&O to address Board requirements and support internal decision making. Its main pillars are the risk & control matrix, a monitoring & evaluation framework, and Local Fund Agent (LFA) assurance. The Secretariat also put significant effort into developing data processes that feed into the framework. The framework goes beyond what was in place for core disease grants and aims to strengthen oversight of the entire portfolio.

Key issues

1. Fast implementation exposed challenges with external review processes: In line with a Board Decisions, an external review by the Grant Approval Committee (GAC) and the COVID-19 Technical Advisory Group (CTAG) partners was established for C19RM 2.0, and met all requirements set in the Board Paper.  Its application led to tensions in implementation: two-thirds (67%) of all GAC-CTAG respondents to the OIG survey said that insufficient time was given to robustly review funding requests, and 25% of respondents were not able to review and submit comments on time.
2. Data quality issues in procurement reporting due to reliance on manual tools and processes: Procurements represent 66% of total C19RM 2.0 investments. As such, monitoring the Pooled Procurement Mechanism (PPM) and local procurement is critical. New reporting processes and requirements were put in place, including country-level processes/ tools and new Secretariat key performance indicators (KPIs). However, data quality issues hindered effective oversight. For local procurement reporting across the ten sampled countries, the amount reported as locally procured was ten times higher than actual. For PPM, reporting issues were noted in the calculation of KPIs that are reported to the Secretariat. These issues are linked to reporting being heavily dependent on excel templates and labour-intensive manual processes to collate and analyse data.
3. Sub-optimal utilization of C19RM 2020 funding, due to in-country procurement and supply chain management issues.
4. Processes to support effective cross-portfolio optimization of funds are needed: These would help to maximize future savings and to respond to significant changes in initial assumptions or changes in the external environment.
5. In future, the current operating model needs significant and careful assessment when responding to acute emergencies.

Audit findings

1. COVID-related funding was reviewed and approved quickly, but at the expense of robustness in some key areas:

1A .C19RM maintained a rapid pace in approving grant requests.

1B. The mechanism had been strengthened when compared with C19RM 1.0.The robustness of the review and approval process was enhanced in: internal finance review, Investment Committee approvals, Board reporting, C19RM integration into existing grants, procurement, disbursements, guidance and tools, and alignment of investment categories.

1C. The process of allocating funds across portfolios has been strengthened. However, there are some limitations in the design of the process and in the data used to inform the qualitative adjustment process.

1D. There was a strong perception of inclusion in the C19RM 2.0 request and approval process. The positive impact of inclusion efforts was noted in the OIG survey of CCM stakeholders:

These are good results compared to C19RM 1.0, when only 56% of CSO/KVP respondents said there had been sufficient consultation:

1E.The External Review process was established and well-executed but fast implementation exposed challenges.

To put this into context, the report explains that External Reviews were conducted by GAC and CTAG partners (including WHO, GAVI, Stop TB and UNICEF) in parallel with the internal reviews performed by Global Fund Secretariat teams. The External Reviews assessed: (a) the technical robustness of applications; (b) the alignment/coordination of investments across donors and national stakeholders; and (c) alignment with WHO technical guidance and national preparedness plans. The overall approach was to: (i) notify External Reviewers that a new Funding Request was ready for review and request them to provide their input within three working days; (ii) share Funding Request documents via an online platform; (iii) collect External Reviewers’ inputs using online tools and consolidate them into a summary that was shared with Country Teams; and (iv) synthesize the External Reviewers’ inputs and Country Team responses and provide the synthesis to the Investment Committee (IC) to inform the award decision.

The implementation challenges noted in the audit report were:

1. The lack of minimum requirements for the GAC-CTAG review which resulted in differences in the quality of the reviews.
2. The technical review process was designed with a 72-hour window, as per the approved Board Decision, emphasizing the speed of review. In implementation, however, this impacted the robustness of the review, something which may not have been foreseen when initially designed. GAC-CTAG respondents to the OIG survey highlighted that there was insufficient time to review funding requests:

1F. For oxygen investments, prioritizing speed and the lack of in-house technical capacity impacted the anticipation of downstream risks.Despite the initial gaps in technical knowledge and understanding, the Secretariat made significant efforts and investments in improving its internal technical understanding of oxygen provision, to enhance support to countries.

1G. There was – and remains – a need to continue to strengthen people, tools, and systems to support C19RM 2021 across its life cycle.

2. The design of the M&O framework is comprehensive, with some initial challenges in roll-out and implementation.  The OIG report provides the following overview of the C19RM reporting, monitoring and assurance structure and timeline and status of the M&O framework:

2A.       As of December 2021, the M&O framework had not been fully implemented (see the figure below); significant components of upstream monitoring and reporting had been in place for several months, however, country analysis and reporting based on results from Pulse Checks and Spot Checks remained in-progress:

2B.       The Board had set out detailed requirements with respect to upstream monitoring and oversight and the Secretariat materially met all these requirements through monthly Board reporting and through developing and utilizing multiple channels to support engagement and knowledge sharing with the Board on C19RM 2.0.

Most Board survey respondents highlighted that they were satisfied with the frequency (88%) and amount (83%) of information provided by the Secretariat on C19RM 2.0. Most respondents (59%) said engagement and transparency had improved compared to C19RM 1.0. However, perspectives relating to the disclosure of operational challenges at the Secretariat and country level were less favourable with the following respondent percentages rating Secretariat disclosure as poor/very poor:

One reason for the less favourable perspectives on disclosure of operational challenges is that while there were clear Board requirements for reporting on progress for the approval phase, there were no prescriptive details around key reporting expectations for the implementation stage. It is also important to highlight that as the M&O framework is not fully operational, there are still gaps in the data received from countries, including on operational challenges.

2C.       The M&E framework’s design is comprehensive but is not yet fully operational.  With data collection at a nascent stage, the OIG could not comprehensively assess the implementation and effectiveness of the M&E framework.

2D.       Mandatory minimum LFA assurance on in-country supply chain was not implemented, leading to limited assurance over system capacity and gaps.

2E.       There are limitations in the oversight of post-IC approval changes for grants.

2F.       There are issues with reporting on procurement activities, leading to gaps in the visibility of progress. Given procurement’s key role within C19RM 2021, a greater emphasis was put on monitoring and tracking both PPM and non-PPM procurements which resulted in new reporting processes and requirements.

2G.       There is no centralized reporting process to track the status of legal requirements and management actions on contingent disbursements.

3. The utilization of COVID-19 funds was slower than expected due to global and country-level challenges, as well as untimely monitoring:

3A.       C19RM 1.0 utilization was not as expected at country level due to procurement bottlenecks.  As of 30 June 2021, use of C19RM 1.0 funding across all portfolios was 47% based on the latest information available, with significant variations between individual countries.

COVID-19 exposes pre-existing challenges with procurement supply management (PSM) at country level.  In six of the OIG sample countries, significant PSM challenges were highlighted as limiting the impact of C19RM 1.0 funding, with three common PSM root causes: (i) global production and supply side factors; (ii) in-country local procurement capacity and processes; and (iii) regulatory and administrative challenges at country level.

3B.       There was limited Secretariat visibility to effectively monitor C19RM 1.0 funds: (a) The lack of visibility on absorption levels hindered mitigation of absorption risk. (b) The inability to isolate C19RM 1.0 funding from grant flexibilities and to obtain detail on procurements impacted monitoring.

3C.       Despite early warning signs from C19RM 1.0 regarding utilization and procurement bottlenecks, significant time was taken to convert an initial request into orders, increasing the risk of under-utilization in the later stages of C19RM 2.0.

4. There are opportunities to improve speed and agility in processes, systems and tools beyond 2021.

4A.       C19RM 2.0 is exposed to more volatile global changes in epidemiology and is reliant on fast-changing, sometimes contradictory information when approval decisions are made.  As a result, as new information is collated and analysed, there is a need to ensure this is quickly and continuously factored into decision making during implementation. The mechanism needs to be more agile than for core grants to respond to changes throughout its life cycle, which is significant as C19RM 2.0 will support implementation until December 2023.

4B.       The lack of a cross-portfolio process is likely to lower C19RM’s agility in years two and three of its life cycle (i.e., during the implementation phase). It impacts the Global Fund’s ability to reprioritize savings in line with regional and global changes in epidemiology and to respond to country-specific challenges around absorption and shifts in the funding landscape. To tackle this, portfolio optimization processes for savings across portfolios need to be defined.  To support this, there is a need to determine the principles of what constitutes savings or new funding, through a new portfolio optimization process.

4C.       Portfolio optimization processes are needed to adapt to low absorption and COVID-19 disruptions.

4D.       There is a need to enhance key systems and tools that support rapid savings identification and portfolio optimization.

4E.       C19RM 2.0 introduced a two-tier approach to accessing funding: the full funding process and the fast-track application process. The purpose of fast-track was to ensure that countries’ urgent needs for COVID-19 commodities were supported, although it was acknowledged that even with immediate order placement, there would be significant lead times before essential commodities arrived in country. Essentially the mechanism was not designed for acute emergency response.

4F.       The low uptake of fast-track funding and lack of differentiation in timelines impacted its overall usefulness.  As of December 2021, in response to 36 applications, $591 million had been awarded through the fast-track approach representing 66% of the available funding that could have been accessed through this method:

Across the 10 OIG sampled countries, the speed of fast track to support countries from initial request to purchase order was 76 days, versus 84 days for full funding. These data highlight an underused fast-track mechanism, as well as some limitations in administering applications quickly.

Country capacity to engage across two types of requests quickly and effectively was noted as a limiting factor to accessing the mechanism in a timely manner. 75% of CCM respondents to the OIG survey highlighted capacity challenges to manage the C19RM 2.0 processes alongside their existing workload.

4G.       There are structural imitations that impact the Global Fund’s ability to support COVID-19 acute emergency response.

COVID-19 has had an overwhelming impact on all stakeholders, highlighting a widespread lack of emergency preparedness to deal with a pandemic of this nature and scale. Differing peer approaches highlight potential further avenues for the Global Fund to explore, with the aim of enhancing C19RM’s speed and agility. But equally, there are numerous trade-offs and risks associated with each approach that would have to be carefully considered. Given the impact of COVID-19 on the global supply chain for key commodities and products, it is unclear if any alternate approach would ensure a faster response, and various peer organizations have faced different challenges due to their models.

Agreed Management Actions (AMAs)

1. Acknowledging the primary role played by in-country partners, the current implementation stage and the feedback from the OIG review, the Secretariat will, by 31 December 2022, review the external review process in order to further optimize and mature the role played by GAC/CTAG.
2. In line with lessons learned from the implementation of the C19RM M&O framework and the Audit findings, the Secretariat will, by 30 June 2023, continue strengthening elements of the M&O framework.
3. The Global Fund Secretariat will, by 31 December 2022, develop and operationalize a portfolio optimization framework for C19RM investments in line with the evolving nature of the COVID-19 pandemic; this will consider factors including absorption to date and the evolution of external funding landscape.

Commentary

This is an interesting and important report that is worthy of reading several times because the processes involved in designing, launching, managing, monitoring and evaluating the C19RM mechanism are complex.  The findings and recommendations will be unsurprising to those who have been associated with aspects of preparing applications and implementation.

One clear message comes across: the C19RM was – and may be needed again – to deal with an emergency whereas the policies and operations of the Global Fund disease grants are not designed for an emergency situation.  There has been a good attempt to modify Global Fund policies, practices and systems to cope with but to deal more effectively with emergencies calls for very different approaches and systems.

On reflection, there may be three aspects that require further attention.  First, the COVID-19 pandemic is not over.  We do not know what may or may not develop; it is therefore a fluid situation.  Further, as yet unforeseen, adjustments may be required.

Second, capacity to implement at the country level is a recurring issue at the best of times.  Given this constraint, more thought must be given as to how emergency pandemic applications can be more easily and quickly formulated.

Third, the due dates for completing the agreed management actions may leave too little time for improvements to have a significant effect on the C19RM 2021 outcomes.  Given the urgency, some effort should be made to advance those actions.

[N.B. The Global Fund’s response to item 3 above was that it is important to bear in mind that the AMA means of verification cover both AMA design and implementation. For example, on AMA 3, the OIG requires Portfolio Optimization to not only be designed but to be already rolled out by the due date. Hence, it would be designed and implemented in Year 2 of the mechanism.]

Despite all the challenges and limitations, the Global Fund must be congratulated on what it was able to achieve with both C19RM 2020 and 2021.  Compared with many peer organisations, it has done a good job under unprecedented circumstances.  Of course, some of the countries implementing their C19RM grants may have another viewpoint. There has been anecdotal evidence on the difficulties experienced with the C19RM grants not being as flexible as they were supposed to be leading to low non-procurement expenditure; but that is a discussion for another article.