AIDSPAN PAPER EXAMINES HOW THE GLOBAL FUND CALCULATES LIVES SAVED
Author:
Kate Macintyre
Article Type:Article Number: 1
Implications for policies and programmes discussed
ABSTRACT A new Aidspan report, written by David McCoy and Nele Jensen, discusses the assumptions of the methodologies used by the Global Fund to quantify lives saved through programmes that it supports.
Aidspan has published a paper critiquing the way in which the Global Fund quantifies lives saved through programmes that it supports. The paper was prepared by Dr David McCoy, a public health physician and senior clinical lecturer at Queen Mary University London, and Nele Jensen, a physician.
The Global Fund has stated in its Results Report (2012) that between 2002 and mid-2012, 8.7 million lives were saved through Global Fund–supported programmes by the combination of the provision of (a) antiretroviral therapy (ART), (b) direct observational therapy, short course (DOTS) for TB and (c) insecticide-treated nets (ITNs) for malaria. The paper by McCoy and Jensen examines how the Global Fund estimates the number of lives saved. The paper describes and examines the modelling exercise, providing a detailed look at why and how the numbers are generated, and describes possible implications for policies and programming.
The authors examine the components of the statistical models to understand how the Global Fund arrived at the 8.7 million figure. The models, which are used not only by the Fund, but also by the World Health Organisation (WHO) and UNAIDS, compare the estimated impact of current service delivery scenarios on mortality with a hypothetical “no treatment, no services” scenario. The difference in the number of deaths that would occur under these two scenarios is interpreted as the number of lives saved. These estimates are based on: (a) the calculated effectiveness of ART, DOTS and ITNs in reducing mortality; (b) mortality rates; and (c) data from Global Fund–supported programmes on the number of people alive on ART, the number of TB patients treated with DOTS, and the number of ITNs distributed.
McCoy and Jensen highlight how elements of the methods used may produce either an under-estimation of the true number of lives saved or an over-estimation. While they acknowledge the imprecise nature of modelling exercises, McCoy and Jensen criticise the Global Fund for not publishing uncertainty ranges associated with the estimated number of lives saved; and for reporting only a single figure for the estimated number of lives saved, instead of providing separate figures for each intervention.
McCoy and Jensen argue that, on balance, assumptions about the effectiveness of ART, DOTS and ITNs – which come from study trials that often produce results under somewhat ideal situations – mean that the methods are likely to produce an over-estimation of the number of lives saved from the three interventions (ART, DOTS and ITNs). On the other hand, the authors note that the Fund’s calculations do not include estimates of the lives saved by other interventions, such as prevention of-mother-to-child transmission, HIV testing and counselling, delivery of male circumcision services, treatment of sexually transmitted infections, promotion of condoms, treatment of malaria and indoor residual spraying.
The paper welcomes the Global Fund’s intention to conduct more direct measures of impact through more focused country-level evaluations in all its high impact countries. This will allow the Global Fund to measure as well as model lives saved. The authors state that this will enable a more refined approach to impact measurement, and that it will likely reveal considerable contextual variation in the effectiveness of ART, DOTS and ITNs, both within and between countries. By learning more about this variation, the authors state, policy makers will be able to focus not only on the delivery of ART, DOTS and ITNs, but also on the conditions under which ART, DOTS and ITNs are most effective.
Another policy-related issue that is discussed in the paper is the consequences of the Global Fund selectively focusing on ART, DOTS and ITNs. The authors state that this may encourage implementers to maximise the delivery of those outputs, and hence encourage what they refer to as the “over-verticalisation” of HIV/AIDS, TB and Malaria programmes. In addition, McCoy and Jensen wonder whether the emphasis on measuring impact might not disadvantage interventions that lack a hard evidence base of impact. Such interventions may take time or operate through more complex and indirect pathways, making them difficult to evaluate. Examples include health systems strengthening investments, such as investing in better medical and nursing schools, and improving health information systems.
Finally, McCoy and Jensen briefly discuss the question of attribution. Presently, the estimation of lives saved from ART, DOTS and ITNs is attributed not to the Global Fund specifically, but to Global Fund–supported programmes. These programmes also receive financial support from governments, other donors and from patients themselves. As a result, the authors note that “while about 45% of global ART results are credited to ‘Global Fund–supported programmes’, the contribution of the Global Fund to total AIDS programme financing in low- and middle-income countries has been estimated to be only 10%.”
McCoy and Jensen argue that results should not be attributable to “Global Fund–supported programmes.” Instead, they propose two ways of measuring and attributing impact. First, measuring impact at the country-wide level and attributing improvements in impact to all relevant actors, including governments and donors. Second attributing impact to individual agencies on the basis of their proportional financial contributions to national budgets for the specific health programmes. The authors maintain that this would present a more accurate picture of the specific contribution of external agencies, while at the same time explicitly recognising the contributions of governments and of cross-cutting health systems investments to disease-based programmes. This, they argue, would also encourage the Global Fund (and ideally, other agencies) to promote the strengthening of coherent and comprehensive national programmes. The authors state that the Global Fund has pointed out this discrepancy.
In a separate comment made to Aidspan, McCoy said: “The Global Fund leads the way in measuring results and performance when compared to other development partners. We hope our paper will advance the discussion of how the Fund can help generate more accurate data, and more appropriate systems of performance evaluation and measurement of impact at the country level that are holistic and include domestic and external agencies.”
A spokesperson for the Global Fund Secretariat said that the Secretariat welcomes the detailed assessment of the way the Global Fund and partners assess lives saved, and welcomes many of the paper’s recommendations. The spokesperson described measures that the Secretariat is planning to improve the quality of its data. These include supporting countries to directly measure (as well as model) impact and lives saved, and developing an impact evaluation plan agreed by all priority countries which will be in place by the end of 2014.
The Aidspan paper, “Quantifying the Global Fund’s Contribution to Saving Lives: Methodological and Policy Issues,” is available on the Aidspan website here.
