Until the Global Fund defines its risk appetite, it can’t know what level of assurance is required: OIG audit

Author:

David Garmaise

Article Type:

Article Number: 4

Assurance activities are insufficiently aligned to highest risk areas

ABSTRACT “The work on defining risk appetite is in its early stage and until [it is] sufficiently advanced, there is limited guidance on the required level of assurance.” So says the Office of the Inspector General in a new audit of in-country assurance. In addition, the OIG said, assurance activities are insufficiently aligned to the highest risk areas in the grant portfolio, particularly supply chain management and programmatic risks. The OIG also noted that programmatic and health product risks have supplanted financial risk in the rankings.

“The work on defining risk appetite is in its early stage and until [it is] sufficiently advanced, there is limited guidance on the required level of assurance.” This is one of main conclusions of an audit by the Office of the Inspector General (OIG) of in-country assurance. It is also one of the reasons why the OIG rated the design of the in-country assurance model as “needing significant improvement,” the second lowest rating in the OIG’s four-tier rating system.

A report on the audit was released on 18 December 2017. This article provides a summary of the report in two sections: (A) a brief overview; and (B) a deeper dive (to use one of the Global Fund’s new buzz words).

1. BRIEF OVERVIEW

Since the last OIG audit on assurance in 2014, the OIG said, the Secretariat has taken several steps to improve assurance, including adopting a differentiated approach for high-risk countries; introducing key risk matrices as a starting point for determining assurance needs; establishing committees to oversee and coordinate assurance activities across different functional areas; and investing in systems and processes to support local fund agent (LFA) management.

“However, challenges remain as the Secretariat is yet to put in place some of the key drivers that underpin effective assurance,” the OIG stated. In addition to the lack of a definition of risk appetite, the OIG reported, “assurance activities are insufficiently aligned to the highest risk areas in the grant portfolio, including supply chain management and programmatic risks.”

The LFA model remains central to the Global Fund’s assurance framework, the OIG stated, but its operational effectiveness is limited by several factors, including the following: an insufficient focus of LFA services on key risk areas; gaps in functional expertise of the LFAs in some of these areas; and the shortage of tools to guide LFA work. In addition, the OIG said, the Global Fund faces a significant concentration risk because the bulk of LFA services remain with one provider.

From a governance and accountability perspective, the OIG stated, although the Risk Department is considered the business owner for assurance, grant-related assurance activities are mainly planned, executed and managed by the Grant Management Division. “Assurance is managed in a siloed manner, by functional areas, which limits the optimization of assurance resources across the full spectrum of the grant lifecycle,” the OIG said. However, the OIG reported, in 2017, the Secretariat started collaboratively managing assurance across the functional areas in the organization.

Table 1 lists the three areas assessed in the audit, the OIG’s ratings and its summary comments for each area.

Table 1: Audit findings at-a-glance

AREA 1: Whether the design of the in-country assurance model and related frameworks is adequate and effective in supporting the identification and mitigation of grant-related risks.	Rating: Needs significant improvement
OIG comments: In the absence of a clear organizational risk appetite framework at the corporate level, assurance remains insufficiently aligned to the critical risks facing the Global Fund. There is insufficient focus on critical programmatic, procurement and supply chain risks as reflected in the work of the key assurance providers. Until these gaps are addressed, there is no reasonable assurance that the in-country assurance model is adequate in supporting the identification and mitigation of grant-related risks.
AREA 2: The adequacy and effectiveness of Secretariat structures and processes in managing assurance to obtain the best value from available resources.	Rating: Partially effective
OIG comments: Whilst Secretariat processes to support the LFA model, including well-established processes over contracting, performance management, conflict of interest review and cost management are adequate, improvements are needed to limit the concentration risk through retendering and rotating LFA services. Ownership and accountability for assurance need to be clarified. The Secretariat needs to better coordinate assurance management.

The OIG has a four-tiered rating scheme, as follows: Effective; partially effective; needs significant improvement; ineffective.

1. DEEPER DIVE

The audit reviewed assurance activities in 15 countries provided by LFAs, principal recipients (PRs), external auditors, internal auditors, health facility assessment providers and country coordinating mechanisms (CCMs). The audit did not involve in-country visits, but leveraged data and observations from other OIG country audits. The 15 countries reviewed as a part of the audit were Kenya, Myanmar, Sudan, Thailand and Viet Nam (all high-impact countries); Guatemala, Namibia, Nepal, Papua New Guinea, Rwanda and Somalia (core countries); and Botswana, Gambia, Georgia and Moldova (focused countries).

 

KEY ACHIEVEMENTS AND GOOD PRACTICES

The OIG listed achievements and good practice in three areas, as follows:

Differentiation. The assurance model has evolved over the years in an effort to keep pace with changes in the organization. Accordingly, the OIG noted, the Secretariat has adjusted the types of assurance sought, introduced different providers, and amended tools. In order to maximize the use of the limited resources available, the Secretariat has applied differentiated assurance measures across its grant portfolio, with an increased focus on high-impact and high-risk countries in terms of the frequency, coverage and scope of assurance.

LFA model. The LFA model “has allowed the Global Fund to oversee a broad range of programs in over 100 countries while remaining lean,” the OIG observed. There are strong management processes to support the LFA model, it added, including for contracting, performance management, conflict of interest review, on-boarding, training and cost management.

Risk management. The Secretariat established an Enterprise Risk Committee (ERC), a high-level management forum to discuss risks and assurance arrangements; and an Operational Risk Committee to review the prioritized risks and mitigations of key country portfolios. In addition, the OIG noted, the Secretariat has established coordination mechanisms under the leadership of the Risk Department to bring together key stakeholders to identify ways in which assurance can be strengthened across different thematic and functional areas.

KEY RISKS

The audit identified four areas of risk, as follows:

• There are gaps in aligning assurance to risks while a risk appetite framework is finalized.
• The operational effectiveness of LFA model is in need of enhancement.
• Oversight and management of assurance by the Secretariat is in need of strengthening.
• Some assurance provider mandates require further clarification.

 

Below, we provide a summary of the audit findings for each area.

Aligning assurance to risks

In the absence of a clearly defined risk appetite, the OIG stated, the Secretariat has developed a differentiation model which covers assurance activities. Under this approach, greater assurance is expected for countries with greatest risk exposure and impact. “However, even this differentiated approach does not fully compensate for the absence of risk appetite,” the OIG said, “as country teams still lack the ability to determine what constitutes reasonable assurance for different types of risks in the various country categories (high impact, core and focused).”

Since the last OIG audit on assurance in 2014, the Secretariat has made progress in identifying risks, the OIG said, but challenges remain in the identification process and how assurance is mapped to those risks.

(The report on the 2014 audit is not a public document. At the time, the policy was not to publish internal audits. This has since changed.)

The OIG said there has been limited assurance over areas that the Global Fund has identified as being riskiest. The highest risks are in programmatic areas and in the delivery of health products and services (see Table 2). However, assurance resources are skewed towards financial risks.

Table 2: Top 10 risks as ranked by Global Fund grants

Rank	% of grants rating as high-risk	Risk	Category
1	46%	Treatment disruptions	Health services & products
2	42%	Poor aid effectiveness & sustainability	Programmatic & performance
3	34%	Poor access & promotion of equity & human rights	Health services & products
4	32%	Poor quality of health services	Health services & products
5	31%	Inadequate M&E & poor data quality	Programmatic & performance
6	28%	Inadequate PR governance & oversight	Governance, oversight & mgmt
7	27%	Theft and diversion of non-financial assets	Financial & fiduciary
8	23%	Low absorption or over-commitment	Financial & fiduciary
9	23%	Poor financial reporting	Financial & fiduciary
10	22%	Not achieving program outcome & impact targets	Programmatic & performance

 

LFAs and external auditors provide the greatest coverage of key risks in the Global Fund grant portfolio, the OIG said. However, in 2015–2016, the period reviewed in this audit, apart from validation of the Progress Update and Disbursement Requests, programmatic, procurement and supply chain management assurance activities accounted for less than 10% each of the total LFA budget.

Since the Secretariat discontinued use of the On-Site Data Verification tool in 2016, the OIG said, limited assurance work has been done on data quality risks in the 11 high-impact and core countries included in the audit sample. The Secretariat did not think that on-site data verification provided sufficient assurance and instead began to rely more on tools recommended by the World Health Organization. In 2016, the OIG reported, the Secretariat provided more detailed guidance around program and data quality assurance. However, the Secretariat is still in the process of adapting and aligning the assurance activities with countries’ risk planning exercises.

There is limited coverage of risks related to poor aid effectiveness and sustainability, access and promotion of equity and human rights, treatment disruption, and quality of services, the OIG recounted. Although the LFA’s health product management specialists cover procurement and supply chain management, the majority of this work focuses on upstream activities such as procurement processes, quantification and forecasting. There is less emphasis on downstream supply chain activities, up to facility level and last mile distribution, the OIG noted, where key risks such as treatment disruption are most likely to materialize.

The Secretariat considers new assessment tools, such as the Health Facility Assessments (HFAs), to be more efficient and better aligned with national processes, the OIG stated. The HFAs are expected to provide assurance over data quality, quality of service, and supply chain risks related to commodity stock-outs and expiries at facility level. However, the OIG observed, the effectiveness of HFAs as a form of assurance may be limited by several factors, such as (a) low frequency (once every two or three years); and (b) limited scope (primarily focused on facilities).

In addition, there have been issues in the operationalization of the HFAs. The assessments, introduced in 2016 by the monitoring evaluation and country analysis team, were significantly delayed and thus not fully operationalized by the Secretariat as planned. Only 15% of the HFA budget for 2016 was spent.

Concerning external auditors, they focus primarily on financial and fiduciary risks.

The Secretariat has recently identified other sources of assurance, including annual thematic reviews on key programmatic areas, country-specific evaluations for each grant cycle (i.e. every three years), and prospective country evaluations to be conducted by the Technical Evaluation Reference Group. As these are just being introduced in 2017, the OIG said, it is too early to assess their effectiveness.

Effectiveness of the LFA model

Gaps in the Secretariat’s prioritization of LFA work

LFA reviews are not always guided by country risk assessments, the OIG said. Tools have consistently identified programmatic, procurement and supply chain management (PSM) as higher risk areas, but this is not reflected in the focus of LFA activities.

In addition, the OIG noted, LFA budgets are under-utilized. In 2015–2016, 18% of the global LFA programmatic assurance budget and 25% of the LFA PSM assurance budget went unspent.

Finally, the OIG observed, the tools that support the LFA reviews have limitations. The tools guiding the work of the LFA on financial risks are quite mature, but those supporting programmatic and PSM reviews are much less developed, the OIG said; and the Progress Update Disbursement Request contains limited information on PSM.

Skills not always aligned with the nature or level of risks

There are challenges related to staffing arrangements, the OIG remarked. Limited availability of competent staff in many countries often results in the LFAs appointing “fly-in” consultants who typically cover multiple country portfolios, the OIG noted; further, PSM and programmatic experts in nearly two-thirds of the 15 countries reviewed were not resident in the country.

The OIG said that criteria for the LFAs need to be reviewed periodically to ensure they remain relevant in light of changing risks. Four of the 15 country teams sampled for this audit raised concerns about the programmatic and strategic capabilities of their LFA teams, the OIG said.

Approved staff are not always available for work, the OIG stated. In some instances, senior staff were budgeted but the work was done by junior specialists.

Deficiencies in management of LFAs

Management of LFAs is done by a central team in the Grant Management Division. Strong systems are in place to manage LFAs, the OIG reported, including performance evaluations, work plans and cost control. However, there some residual issues concerning the LFA tendering process and the Global Fund’s reliance on a small number of service providers.

A 2007 Board decision required that there be a global re-tender of LFA contracts every four years. The OIG reported that the Secretariat encountered problems implementing this decision. Over the years, several different approaches have been used instead. The Secretariat recently proposed that the 2007 Board decision be replaced with a rolling tendering process, and the Board has agreed (see separate GFO article in this issue).

“The approach adopted by the Secretariat considers various criteria for re-tendering and has been less complex to administer than the global tender….,” the OIG stated. “However, it has not resulted in meaningful competition in the provision of LFA services, even though the market was tested in many countries.” The OIG said that 10 out of the 15 countries reviewed have not re-tendered in the past four years and that nine of the 15 countries have had the same LFA service provider for 10 or more years.

The Board’s 2007 decision was intended to encourage diversity of participants in the LFA tendering process. But it hasn’t worked out that way, the OIG noted. The leading service provider held 63% of the 2016 LFA budget and 58% of the country portfolios.

“While the predominance of one service provider has advantages related to simplified vendor management, coordination of activities and knowledge-sharing, it also exposes the Global Fund to a high level of concentration risk,” the OIG said. “The high degree of dependence increases the risk of service disruption in the event that the firm becomes unable or unwilling to provide its services.”

The OIG recognized, however, that achieving the right balance is not easy. “The potential benefits of diversifying the LFA supplier base need to be carefully evaluated against the overarching goal of maintaining or improving service quality and effectively managing costs” it stated.

Oversight and management of assurance

As part of strengthening the Global Fund’s risk oversight framework, the OIG said, the Secretariat has steadily improved the level of coordination across the various functional areas involved in assurance. However, assurance is still managed to a large extent in a siloed manner. “Roles and accountabilities are, in many cases, insufficiently clarified, thus limiting accountability,” the OIG observed.

Country teams are responsible for ensuring that implementers have established effective controls to mitigate program risks. The teams also manage assurance processes to evaluate the quality of program implementation and the effectiveness of risk mitigation measures – especially with respect to the LFA. On the one hand, the OIG said, the predominance of the country teams in the assurance process appropriately reflects their role as first-line owners of risk. On the other hand, it said, there is also “an inherent vulnerability” in that the entity that oversees program implementation –– the country team –– also plays a major role in providing assurance about how effectively the risks in program implementation are being managed.

This vulnerability can be substantially mitigated by having a strong second-line oversight function, the OIG stated, such as the Risk Department or the Finance Division. In the Global Fund, second-line oversight has traditionally been effective in managing financial and fiduciary risk, but less effective in overseeing programmatic, procurement and supply chain risks, the OIG observed.

The Secretariat has put systems in place for planning, coordinating and managing assurance, the OIG noted, but challenges persist. Under the Global Fund’s accountability framework, ownership for assurance activities is spread among six departments and teams at the Secretariat, all with separate budgets, guidelines and functional priorities. This makes it difficult to effectively coordinate and manage a comprehensive assurance approach, the OIG stated.

As mentioned above, key risk matrices have been introduced as part of the planning process. However, the OIG said, country teams do not always implement the assurance actions agreed in the matrix because the process of developing assurance plans as part of the key risk matrices is independent of the annual LFA planning process. Furthermore, the OIG said, even for the sampled countries that do not have key risk matrices yet (11 out of 15), there was limited evidence that risk assessments were driving the allocation of LFA resources in the annual LFA planning process. This represents a missed opportunity to tailor assurance to areas of greatest risk, the OIG concluded.

Since the start of 2017, the OIG noted, there has been progress in coordination with the creation of a Risk Management Group which holds monthly coordination meetings between functional areas.

Assurance provider mandates

The mandates of external auditors and LFAs are clearly defined through the grant agreements and specific terms of reference. However, the OIG noted, there are gaps with other providers.

Oversight mechanisms in-country, such as the PR’s monitoring functions and internal audits, could provide additional assurance, the OIG said; however, only limited reliance can be placed on these functions, in part due to a lack of clearly defined mandates. In general, the OIG said, grant implementer internal audit functions tend to lack effective charters, clear terms of reference and adequate reporting lines; these limitations, combined with generally tight resources, significantly constrain their effectiveness. Likewise, the OIG observed, although country coordinating mechanism (CCM) oversight activities provide some level of assurance, CCMs were established for different purposes and, as such, they do not have a clear assurance mandate. The OIG noted that the work of the CCM evolution project may lead to better clarification of the CCM’s oversight role.

The terms of reference for external auditors and for LFAs are generally focused on transactional verifications and do not emphasize the review of internal controls, unlike terms of reference used by other donor organizations, the OIG said. For example, the terms of reference for the external audits of USAID recipients require the external auditors to provide a specific report on internal controls. The terms of reference used by the World Bank require auditors to provide an opinion on the adequacy of the internal control structure of the project.

In the case of the Global Fund, however, the current guidelines for annual audits of the grant program financial statements do not require an assessment of the implementer’s internal controls, the OIG stated.

PROGRESS ON PREVIOUSLY IDENTIFIED ISSUES

The 2014 OIG audit assessed the extent to which the Board and senior management could rely on the work of first- and second-line assurance providers. The audit concluded that while the Secretariat had instituted several measures to strengthen its assurance mechanisms since the 2011 High-Level Panel report’s findings on assurance, coordination of different assurance providers and management of information produced by them were still challenging. The Secretariat undertook to set up a Combined Assurance Working Group to inculcate a culture of accountability and transparency at the Global Fund.

Two initiatives were born out of this working group: (a) the Risk and Assurance Project (driven by the Risk Department); and (b) a project to establish guidelines for financial assurance planning for grants (in the Finance Department). These initiatives helped resolve some of the assurance issues identified. However, the working group was discontinued before it could achieve its overarching objective and a replacement working group was set up in late 2016.