HEALTH IMPACT EVALUATION – MAJOR FINDINGS

Author:

Bernard Rivers

Article Type:

Article Number: 2

ABSTRACT The rapid increase in funding from all partners for the three diseases has resulted in a major expansion in access to services in these countries and has improved coverage of interventions, which will likely impact disease burden. However, gaps in the availability of data prevented the evaluation from actually measuring the impact of the Global Fund and its partners on the three diseases. These are the major findings from the Health Impact Evaluation.

Editor’s Note: This article summarises the major findings of Study Area 3 of the Global Fund Five-Year Evaluation – Impact on HIV, tuberculosis and malaria, also referred to as the “Health Impact Evaluation.” It is based on the report entitled “Technical Evaluation Reference Group Summary Paper: Study Area 3 – Health Impact of Scaling Up Against HIV, Tuberculosis and Malaria: Evaluation of the Current Situation and Trends in 18 Countries” (hereinafter Summary Paper: Study Area 3), which summarises the Study Area 3 report of the international evaluators (see previous article), and provides an assessment of that report by the TERG.

The rapid increase in funding from all partners for the three diseases has resulted in a major expansion in access to services in these countries and has improved coverage of interventions, which will likely impact disease burden. However, gaps in the availability of data prevented the evaluation from actually measuring the impact of the Global Fund and its partners on the three diseases.

These are the main conclusions from the Health Impact Evaluation, the third and last study area in the Global Fund Five-Year Evaluation.

The objective of the Health Impact Evaluation was to comprehensively assess, in selected countries, the collective impact that the Global Fund and other international and national partners have achieved on reducing the disease burden of AIDS, TB and malaria.

The study was conducted in 18 countries – 10 where the evaluation was based primarily on existing information (Benin, Burundi, Democratic Republic of Congo, Ghana, Kyrgyz Republic, Lesotho, Moldova, Mozambique, Rwanda and Viet Nam) and eight where additional data was collected specifically for the evaluation (Burkina Faso, Cambodia, Ethiopia, Haiti, Malawi, Peru, Tanzania and Zambia).

The study methodology was designed to document the trends in the three diseases, including mortality and morbidity. As part of the methodology, country-level Impact Evaluation Task Forces were established in each participating country. These Task Forces had broad representation from local institutions, including Ministries of Health, civil society, CCMs and donors. The Task Forces reviewed the country evaluation work plans and budgets, as well as draft and final country reports.

In total, $11.7 million was spent on the Health Impact Evaluation, 40 percent of which was for data collection.

Key findings

The following is a summary of the key findings of the study, as reported by the evaluators. These findings relate to the 18 countries included in the study; they do not necessarily reflect the situation in all countries that receive Global Fund grants.

Impact on HIV/AIDS

• HIV funding has increased rapidly. Eighteen percent of the additional money came from the Global Fund.
• There has been a major expansion in access to services in all countries. However, there are gaps in basic requirements – such as trained personnel, guidelines, medicines, and equipment – that negatively impact the quality of services.
• There have been dramatic increases in estimated coverage of antiretroviral (ARV) treatment.
• There have been reductions in HIV high-risk behaviours among men in the general population in most of the countries.
• There is little evidence of large-scale changes in behaviours among the most at-risk populations.

Impact on TB

• Positive trends in treatment success rates have continued in most countries, but there is little evidence of accelerated progress since 2003 (when scaling up began in earnest).
• There is widespread access to TB services, although there are no major increases since scaling up.
• There is considerable scope for improving the quality of diagnostic and treatment services.
• TB notification rates are stable or declining in several countries, but the required supporting data on diagnostic intensity is often lacking.

Impact on malaria

• There have been major increases in funding, led by the Global Fund.
• Malaria diagnostic capacity remains suboptimal.
• The availability of artemisinin-based combination therapy (ACT) in most countries is limited.
• Major progress has been made in coverage with insecticide-treated bed nets and intermittent preventive treatment of malaria during pregnancy.

Impact on health systems strengthening (HSS)

• Health worker density is low in all districts, especially in rural areas.
• In many facilities, there are serious deficiencies in terms of basic amenities, especially improved water supply and essential equipment. The situation is somewhat better in facilities that offer HIV services.
• There is inadequate availability of many essential medicines, especially for chronic adult diseases but also for childhood illnesses.
• There are major gaps in the availability of diagnostics.

The evaluators found that although there was evidence of increased data collection for the three diseases, major data gaps and weak health information systems remain. These gaps and weaknesses seriously limit the ability to evaluate progress.

The consultant team commented that because investment in evaluation by the Global Fund and its partners over the years has been limited, it was only possible to partially document trends in the three diseases – and, therefore, only possible to partially assess the impact of the Global Fund and its partners.

Comments by the TERG

In it assessment of the study carried out by the evaluators, the TERG said:

• The Health Impact Evaluation was exceptionally challenging from both a methodological and practical point of view.
• The design of the study, as submitted to the Global Fund Board in 2006, recognised that certain questions likely could not be fully answered.
• The study identified major gaps in availability of data and greater health information system weaknesses than originally predicted.
• There were inherent tensions between the desire for inclusive participation and country capacity building, on the one hand, and the requirement for high-quality evaluation products that had to be delivered within a short time frame, on the other.
• A recurrent issue in the evaluation is the absence of solid and consistent baseline data upon which to base conclusions regarding the effects of scaling up.

The TERG said that to address the information gaps, additional studies are needed in the following areas:

• civil society and community-based interventions
• non-health-based HIV prevention interventions
• interventions targeting high-risk groups
• differences in performance between countries
• effectiveness of specific interventions
• HIV/TB co-infection interventions

In conclusion, the TERG cited the following comment from one of the persons who reviewed a draft of the evaluation report:

“In many respects, this evaluation process shares many of the characteristics of the Global Fund itself. It was conceived with the right principles and approach in mind, along with engaging the best technical people and giving them at least reasonable financial resources to initiate an innovative process. The technical team developed a thoughtful and, in most respects, state-of-the-art approach towards tackling the problem. However, this evaluation faced significant challenges once it entered the real world of extremely weak country institutions, multiple stakeholders with poor in-country coordination, and very poor routine information systems.”

Nevertheless, the TERG said, the Health Impact Evaluation was worthwhile because it produced useful data in 18 countries, and it has helped to strengthen the foundation for future impact assessments.