Subscribe To Our Newsletter
Abonnez-vous à notre bulletin
Strategy Committee discusses an update on evaluation matters and the 2024 work plan and budget
GFO issue 439

Strategy Committee discusses an update on evaluation matters and the 2024 work plan and budget

Author:

Aidspan

Article Type:
News

Article Number: 5

At last week’s Strategy Committee meeting, one of the discussions centred on the Evaluation Matters Update and the 2024 workplan and budget for the Evaluation and Learning Office, for SC decision.

 

The Strategy Committee (SC) met on 9-11 October and one of many issues discussed for information was an update on evaluation matters. This was followed by the SC decision on the annual work plan and budget for the Evaluation and Learning Office (ELO) for 2024.

 

Context

 

The Secretariat presented a joint update between the Independent Evaluation Panel (IEP) and Secretariat ELO. IEP and ELO have advanced significantly on operationalizing the evaluation function as set out in the Board decision in November 2021 (GF/B46/DP06). It noted that further alignment was required on a small but critical subset of issues.

 

In November 2022 the Board approved a multi-year evaluation calendar (GF/B48/04 Annex 3) which formed the basis for the 2023 evaluation work plan and has subsequently informed the 2024 work plan for 2024 being presented at this SC for decision.

 

Throughout 2023, IEP and the Chief Evaluation and Learning Officer (CELO) have engaged in extensive consultations, including liaising with SC Leadership, to operationalize the evaluation function to ensure adherence to various terms of reference and governance charters. Consultations have resulted in the development of draft set of standard operating procedures (SOPs).

 

Update on 2023 Evaluation Work Plan and evaluations in progress

 

Table 1. Implementation of the 2023 Work Plan

 

Evaluations

  • Two high-priority evaluations were launched in 2023 and will be finalized in early 2024. One is the Strategic Review 2017-2022 (SR 2023), now in the data collection and analysis phase, and the other is on Allocation Methodology which is in its inception phase.
  • The first round of a country stakeholder feedback mechanism, Imbizo (a Xhosa word meaning ‘a gathering to share knowledge’) will be launched end 2023.
  • Due to staff capacity and other start up work, no fourth evaluation will be launched in 2023.

IEP

  • IEP approved the Terms of Reference for the ongoing evaluations, appointed a focal point for quality assurance for each evaluation and developed a draft framework for quality assessment of evaluation reports.
  • Two new IEP members were recruited to replace the Technical Evaluation Reference Group’s (TERG) outgoing former chair and vice-chair following the transition period to the new evaluation function.
  • First IEP annual report to be submitted to the Board through the SC in March 2024.

Operational shifts and procedures

  • Evaluation eligibility criteria developed and applied to 2024 topics proposed and will be used moving forwards.
  • Evaluation ‘Engagement teams’ composed of ELO staff and IEP focal points have been, and will be, formed for each evaluation. New mechanism established to solicit inputs from key stakeholders responsible for acting on findings.
  • ELO and Office of the Inspector General (OIG) staff have worked closely together to mitigate overlaps on topics and in data collection.
  • Vendor mapping process was initiated to expand the pool of evaluators and firms.
  • TERG document procedure under revision for the new evaluation function and ensure the continued transparency and publication of evaluation evidence. To be submitted for decision to SC in March 2024.

 

The four figures below depict overviews of the 2023 evaluation schedule plus the SR 2023, Allocation Methodology and Imbizo.

 

Figure 1. Overview of the 2023 evaluations

 

 

 

Figure 2. Overview of Strategic Review 2023 (SR 2023)

 

 

Figure 3. Overview of Allocation Methodology

 

 

 

Figure 4. Overview of Imbizo

 

 

 

Update on SOP development and overview of key roles and responsibilities

 

Two draft SOPs have been developed by the ELO under the oversight of the IEP:

 

  • SOP 1: outlines the annual process for identifying and prioritizing evaluation topics, developing and launching the Annual Evaluation Work Plan and making any modifications to the Multi-Year Evaluation Calendar.
  • SOP 2: outlines the process for the end-to-end management of an individual evaluation.

 

The SOPs are aligned to key GF documents

 

The SOPs are designed to align with the CELO’s job description, the IEP’s Terms of Reference, and the Board and SC oversight role as described in the relevant Governance Charters. In developing the SOPs, attention has been focused on: how independence of the evaluation function can be safeguarded; how oversight leading to high quality evaluations can be achieved; and how learning and utility can be advanced. The high-level roles and responsibilities of stakeholders outlined in the SOP include Governance, IEP, ELO and the Secretariat and are presented in summary tables as follows.

 

Table 2. SOP1 – Annual Evaluation Plan – Engagement Overview

 

 

 Table 3. SOP2 – Evaluation End-to-End Process – Engagement Overview

 

 

Evaluation workplan and budget

 

Following the aforementioned Board approval of a new independent evaluation function in November 2021, the SC has the delegated authority to approve the strategic priorities and the annual workplan for the evaluation function of the Secretariat. The Evaluation Function includes the activities of two structures responsible for delivery of the Evaluation Function, the Secretariat ELO and the IEP.

 

In November 2022, the Board approved a Multi-Year Evaluation Calendar  as part of the Monitoring and Evaluation Framework for the 2023-2028 Strategy. The Evaluation Calendar forms the basis of the evaluation function annual workplans and the first workplan for the new evaluation function was approved by the SC in November 2022.

 

The 2024 evaluation annual workplan and budget includes ongoing and new evaluations identified as per the Evaluation Calendar and described above. It also includes the oversight by the IEP, implementation of innovative learning and dissemination activities for evaluations completed in 2023 while finalizing operational procedures and building functional capabilities for the Global Fund for the longer term.

 

The Secretariat presented the high-level activities of the Evaluation Function in 2024.

 

In 2024 the ELO, under IEP oversight, plans to finalize the three evaluation activities (Strategic Review 2023, Allocation Methodology, Imbizo (formerly called a Country-steered Review) launched in 2023 and will conduct up to four more thematic evaluations. The ELO and IEP will formalize operational procedures and continue to advance the learning function and strengthen evaluation practices and partnerships. To allow adequate opportunity for collaboration, the ELO and IEP will meet regularly; this includes two in-person meetings during 2024 and additional virtual meetings as required, as well as regular coordination meetings between the IEP Leadership and CELO. IEP focal points will continue to engage in quality assessment and assurance.

 

Stakeholder feedback

 

People commended the Secretariat for the clarity and precision of the documents presented, as well as the progress made in the three important ongoing evaluations. They were particularly pleased to see the new Imbizo Initiative taking shape. They were also happy to see the progress on specification of the SOPs (1 and 2) of the evaluation function. As clearly laid out in the engagement overview tables, the steps and responsibilities appear to have been well-conceived. Stakeholders agreed that the end-to-end process should provide adequate opportunity for the IEP to provide input and quality assurance with independence and hoped that sufficient time would be given to the IEP to do so.

 

Stakeholders said that the decisions on the Evaluation Work Plan and Budget and the Evaluation Function Document Procedure are important to continue discussions and development of clear strategic and data-driven approaches to guide implementation. They will count on those evaluations to lead to the successful implementation of the Global Fund Strategy 2023-2028, whether it be on: (i) RSSH and human rights-driven actions as for the new evaluations planned on engagement with the communities and community health systems strengthening; or (ii) a more sustainable approach to health with the evaluation on malaria as an example for the need for an integrated One Health approach and optimized health- and climate-related actions; or (iii) ongoing and future funding requests/grant management.

 

Stakeholders offered several key comments on the Details of the Evaluation Topics for 2024, notably:

 

  1. The second indicative evaluation question under the Funding Request/Grant Making evaluation, as written, explores the positive effects of the “grant design” process. Could stakeholders assume that the evaluation would also explore and document any process factors that may hamper prioritization of certain topics such as RSSH, HIV prevention, etc. and stifle change and innovation?
  2. People would be interested to know the rationale for the narrow scope of the malaria evaluation. While sub-national data completeness, quality and analysis are important to the understanding of local malaria epidemiology and for stratification and tailoring of interventions, this is only one of the many challenges facing malaria control and elimination. Would it make sense to take advantage of this evaluation to also assess one or more other dimensions of the followed progress and responses to it?

 

Leave a Reply

Your email address will not be published.

Aidspan

Categories*

Loading
Aidspan

Categories*

Loading