Evaluating Large-Scale Health Programs

We want to provide you some information about our course “Measuring and Modeling Impact in evaluations”. The purpose of this course is to give you a better understanding of different measures of impact that could be used in the evaluation of a program in the areas of maternal and child health and nutrition. For each of the measures presented, we will discuss current sources of data you might draw on as well as describe the methods that can be used to measure these. When we describe the methods, we also try to identify the strengths and weakness of the methods as well as their suitably for use in an evaluation. The course also discusses how modeling can be used in evaluations as either a replacement for measuring impact or to supplement measured impact. The last two lessons in this course focus on giving you an introduction and training on how the Lives Saved Tool (LiST) works and how to use it. This model can be used to estimate most – if not all – of the impact measures we describe in the course and can be an important part of both planning and estimating impact in an evaluation of a large-scale program. While this course is self-contained, it is also linked to other courses on evaluation. We developed this course for public health program managers and evaluators and assume the students in the course will have a background in public health with a focus on maternal and child health in low- and middle-income countries. The development of this course was supported by a grant from Government Affairs Canada (GAC) for the Real Accountability, Data Analysis for Results (RADAR) project.

This course provides an introduction to evaluating public health programs at scale. This course focuses on evaluating public health programs and policies in low- and middle-income countries, however, core skills of designing and carrying out an evaluation are applicable to any public health programs and policies. The course will equip you with skills to: 1. Critique an evaluation of an international health program, identifying its strengths and possible weaknesses and how they could be addressed. 2. Develop a technically-sound evaluation plan for a reproductive, maternal, newborn, child health (RMNCAH) and nutrition program being implemented at scale in a low- or middle-income country, including evaluation design, key indicators, measurement methods, analysis, and communication of results. 3. Guide program managers and donors through a process of agreeing on priority evaluation activities included in an evaluation plan for a specific RMNCAH and nutrition program. 4. Make informed decisions about whether they want to pursue further learning and/or a professional role as an evaluator of large-scale programs. The development of this course was supported by a grant from Government Affairs Canada (GAC) for the Real Accountability, Data Analysis for Results (RADAR) project.

This course provides an introduction to household surveys for program evaluation in low-and middle-income countries. The course will equip you with skills to: 1. Explain what coverage is, why it’s important in evaluations, and how it is measured 2. Describe what household surveys can and cannot measure 3. Plan, implement, and analyze household survey, including: 4. Calculate an appropriate household survey sample size 5. Explain the resources required for a household survey 6. Identify an appropriate sampling design 7. Design a questionnaire 8. Describe challenges for coverage survey planning and implementation 9. Present and describe how the RADAR tool can be used The development of this course was supported by a grant from Government Affairs Canada (GAC) for the Real Accountability, Data Analysis for Results (RADAR) project.

This course provides an introduction to designing and evaluating to strengthen the health system. After successful completion of all learning activities, course participants will be able to: 1. Define health systems strengthening 2. Describe health systems frameworks and how to incorporate them into evaluation planning 3. Describe how to design and prioritize implementation of health systems programs 4. Detail approaches to evaluate health systems programs, including data sources and study design 5. Introduce tools to assist with evaluation planning and simple models to assess health system intervention impact 6. Describe how to interpret health system evaluation findings and sustainability and scalability implications The development of this course was supported by a grant from Government Affairs Canada (GAC) for the Real Accountability, Data Analysis for Results (RADAR) project.

This course provides in-depth knowledge about implementation strength, quality of care, and service utilization, which are essential components of health program delivery. This course is primarily aimed at implementers, managers, funders, and evaluators of health programs in low- and middle-income settings (LMISs) targeting women and children, and undergraduate and graduate students in health-related fields. Those who complete this course successfully will be able to: 1. Explain why assessing health program delivery is an essential part of any large-scale program evaluation in LMISs. 2. Design an assessment of delivery for a health program in a LMIS, including proposed documentation of the program, indicators, measurement methods and tools, relevant contextual factors, and a plan for data analysis. 3. Interpret assessment results in the context of a large-scale effectiveness evaluation, and explain how they can be used to improve program policies and implementation. 4. Describe the RADAR tools for assessing implementation strength and the quality of care, and access guidelines for the use of the tools. The development of this course was supported by a grant from Government Affairs Canada (GAC) for the Real Accountability, Data Analysis for Results (RADAR) project.