GAO REPORT FINDS PROBLEMS REGARDING PERFORMANCE AND ASSESSMENT OF LFAS
Author:
Bernard Rivers
Article Type:Article Number: 3
ABSTRACT Local Fund Agents perform work of very variable quality, yet the Fund provides little guidance regarding what it expects of them, according to a report by the US Government Accountability Office.
Local Fund Agents (LFAs) perform work of very variable quality, yet the Fund provides little guidance regarding what it expects of them, according to a report released last month by the US Government Accountability Office (GAO).
In its fourth report on the Global Fund since 2002, the GAO examined three particular aspects of the Fund’s work. First, it reviewed how effectively the Fund oversees the performance of LFAs. Second, it reviewed the Fund’s progress in implementing an early warning system. Third, it reviewed the documentation used by the Fund to support its performance-based funding decisions. The Global Fund accepted the report’s findings and recommendations.
The most important section of the report dealt with LFAs. Because all of the Fund’s staff are based in Geneva, the Fund pays companies that have offices in recipient countries to monitor grant progress. A total of six companies play this “Local Fund Agent” role, with one being chosen in each country. The companies are PricewaterhouseCoopers (PWC), KPMG, Emerging Markets Group, Crown Agents, the Swiss Tropical Institute, and the United Nations Office for Project Services (UNOPS).
PWC and KPMG, the two main LFAs, use different approaches. One uses a decentralized approach in which the majority of the work occurs in-country, and the other uses a centralized approach in which the country teams perform the oversight tasks, but all their reports are vetted by a central team before reaching the Global Fund.
According to the GAO report, numerous people have complained about the quality of grant monitoring and reporting provided to the Fund by LFAs. In the worst cases, this has led to two or three LFA country team subcontracts being terminated each year.
The report says that Global Fund staff cited multiple instances in which LFAs demonstrated lack of capacity. In a recent internal survey among Global Fund staff who deal with LFAs, about half of the staff felt that LFAs with whom they had worked did not provide adequate expertise on health and program issues, and about three-quarters felt that within LFA firms there are substantial differences in service quality across countries. UNAIDS officials added that they have serious doubts about the capacity of LFAs to monitor program progress, specifically noting that LFAs have little understanding of how to weigh and balance various performance targets that are not of equal importance.
Compounding this problem, the Fund has no systematic approach for assessing the quality of LFA services or for identifying when greater oversight of LFAs by the Global Fund is required. Instead, in contrast to the strict monitoring and evaluation that the Fund requires of grant recipients, the Fund assesses LFA performance only informally and irregularly, and it does not systematically document its findings.
In addition to complaints being raised regarding LFA performance, complaints have also been raised by LFAs about the Fund. For instance, as of March 2007, the Global Fund had not provided its fund portfolio managers with any specific training or written guidance defining acceptable LFA performance or reports. As a result, LFAs have found that while one portfolio manager at the Fund might expect them to focus on, say, financial performance by the PR, another portfolio manager might expect them to focus on programmatic performance by Sub-Recipients. As a result, LFAs have found it difficult to establish best practices for their country teams.
The report recommended that the Fund establish standardized expectations for LFA performance, and systematically assess LFA performance. (In a separate development, contracts for current LFAs are coming to an end, and the Fund is inviting current LFAs and new companies to bid on new LFA contracts. See www.theglobalfund.org/en/business%5Fopportunities/lfa.)
The second area covered in the GAO report dealt with the Fund’s progress in implementing a “risk assessment model” to assess which types of grant are in most danger of under-performing, and an “early warning system” to use such a model to highlight specific grants that need particular attention.
The Fund tried this in 2005, but without success. The partially-developed 2005 risk model dealt primarily with quantitative data, both at the grant level (grant size, grant performance, etc.) and at the country level (development indicators, corruption indicators, etc.). The model was supposed to generate reports that indicated potential problems, so that portfolio managers could then evaluate the reports in the light of contextual information. However, the framework was not implemented because it relied too heavily on quantitative indicators and did not adequately account for contextual factors, leading to risk ratings that were too inaccurate to be useful. For example, in a test of the model, a particular grant was rated as low risk based on the host country’s development and corruption indicators, yet the grant ultimately failed and was not renewed for Phase 2.
The report quotes an un-named senior manager from one of the two main LFAs, saying in effect that if the Fund had been successful in assessing grants as high, medium, or low risk, LFAs could have adjusted the amount of oversight they provided to different grants, concentrating most on where there was potential misuse of grant money or limited recipient capacity.
The report says that the Fund is now putting in place the first components of its Early Alert and Response System (EARS), which it is hoped will succeed where the 2005 system failed.
The third section of the GAO report reviewed the documentation used by the Fund to support its performance-based funding decisions.
In its 2005 report, the GAO complained that the Fund’s files provided little explanation of how it made decisions to approve particular grant disbursements or grant renewals, even when grant recipients said they had not met their targets or when data provided by recipients was of poor quality or incomplete.
In its 2007 review, the GAO found that this problem has been adequately dealt with.
The GAO report, entitled “Global Health: Global Fund to Fight AIDS, TB and Malaria Has Improved Its Documentation of Funding Decisions but Needs Standardized Oversight Expectations and Assessments” is accessible at www.gao.gov/cgi-bin/getrpt?GAO-07-627.
