Preliminary results have been released from a civil society shadow reporting initiative on the performance of nine African country coordinating mechanisms (CCMs). The research is being led by the Regional Platform for Communication and Coordination for Anglophone Africa, hosted by the Eastern Africa National Networks of AIDS Service Organizations (EANNASO), in partnership with AIDS Accountability International (AAI) and the CCM Hub at the Global Fund Secretariat in Geneva.

A recent CCM audit released by the Office of the Inspector General (OIG) found significant weaknesses in how CCMs are managed, and how CCMs coordinate and oversee grants (see GFO article). In light of the OIG CCM audit, and the enhanced role of CCMs in the funding model, the shadow report initiative responds to the need for a wide range of stakeholders to be empowered to demand improved CCM performance.

The research was conducted between December 2016 and January 2017 in nine countries: Ghana, Kenya, Malawi, Nigeria, Rwanda, Swaziland, Tanzania, Uganda and Zambia. Research teams made up of local civil society watchdogs developed and implemented a shadow CCM eligibility and performance assessment (EPA) tool, to compare findings with the EPAs conducted by the Global Fund Secretariat. The results will be synthesized into a CCM Scorecard.

The Global Fund EPA is based on six  minimum requirements which all CCMs must meet in order to be eligible for funding: transparent and inclusive funding request development, open and transparent principal recipient selection, oversight planning and implementation, CCM membership of affected communities, processes for electing non-government CCM member and management of conflict of interest. The assessment is conducted through an external evaluation process, after which all CCMs must develop a performance improvement plan based on the results. The civil society shadow EPA contains many of the same questions, with some additional elements around the quality of representation and participation.

The preliminary results from the shadow reports reveal several stark inconsistencies between civil society and the Global Fund assessments. “The biggest discrepancy was around oversight,” says Phillipa Tucker, Co-Founder and Research and Communications Director at AAI. “Geneva marks country coordinating mechanisms as compliant whereas civil society ranks them a lot lower. The watchdogs also rate CCMs much lower around issues of conflict of interest. Those are the big ones.” The figure below shows the shadow report results on these two issues.

Figure: Watchdog ratings of oversight (left) and conflict of interest (right) on nine African CCMs

According to the watchdogs, conflict of interest on the CCM is “dealt” with bureaucratically, but not in a practical sense. They report that good governance is required to manage these conflicts in an open and transparent manner, but in most cases these conflicts are rarely stated and almost never addressed.

But the watchdogs are not always the harsher critic. Tucker notes that when it comes to questions around civil society representation, participation, and accountability to constituencies, the watchdogs mark the CCMs more favourably than Geneva. “In Geneva, there’s an analysis of civil society that’s maybe not as respectful as it should be, or as accurate as it should be,” she said. “They’re quite hard on civil society, and maybe that’s coming through in their assessments.”

Tucker also told Aidspan that there was wide variation in how critical the local watchdogs were of their CCMs. For instance, in Tanzania, the watchdog reports never rated the CCM performance lower than Geneva did. Conversely, in Kenya, Swaziland and Zambia, the watchdogs rated their CCMs far lower than the Global Fund – across all indicators.

According to Prosper Byonanebye, part of the local watchdog team in Uganda, the participatory research design was a significant benefit. “What was supposed to be research turned out to be a capacity building model,” he said. “Most of the people who are directly affected by the Global Fund resources – who have been kept out of the room – had an opportunity to get in the room and get to know what the Global Fund is all about.”

Olayide Akanni with the Nigerian shadow reporting team also credits the process with building local research capacity.  “The research process was quite intensive,” says Akanni. “We had focus group discussions with CCM members, guided by a survey tool of more than 30 questions. Then, we had focus group discussions with non-CCM members, which was a further 28 questions.”

EANNASO, AAI and the CCM Hub have made the raw data from the shadow assessments public and free to access, encouraging interested parties to perform their own meta-analyses of the information under a creative commons license. The Global Fund’s EPAs are not made public, something which Aidspan has drawn attention to as a gap in the Fund’s commitment to transparency (see GFO commentary).

“The shadow reports aim to change the way that the analysis of the performance of CCMs is done,” says Tucker. “The Global Fund’s EPA is a rather top-down, box ticking analysis and it doesn’t actually reflect the subtleties and nuances people are experiencing at country level.” As a community monitoring initiative, the civil society shadow reports give depth and contour to the existing CCM performance data.

The final country reports and synthesis scorecard will be presented to the Global Fund in April 2017. “We definitely expect that there’s going to be buy-in into the results,” concludes Tucker.