Health policy is almost always driven by conventional wisdom. This week’s Thought Leadership piece comes from Emily Elizabeth Crawford, the World Bank’s Development Impact Evaluation (DIME) Nigeria Field Coordinator, who writes about how the use of impact evaluations can help develop better evidence for decision makers in the health sector using examples from evaluations conducted in Nigeria. An earlier version of this article appeared on the World Bank blog. The majority of the photos in the blog have been used with permission from EpiAFRIC, a public health consultancy based in Abuja, which conducts impact evaluations of health programmes.
“Giving midwives monetary incentives based on their attendance is effective in reducing midwife attrition, keeping more of them on the job, and increasing the delivery of babies in healthcare centres. Non-monetary incentives, however, did not have any effect, either by themselves or when provided alongside monetary incentives.”
This was the conclusion of just one of the projects presented at a recent event hosted by the World Bank; Beyond the Status Quo: Using Impact Evaluation for Innovation in Health Policy in Washington, DC. This event featured the results of six health-sector impact evaluations from Nigeria, many of which were completed under a partnership between the Federal Ministry of Health (FMOH), the Bill and Melinda Gates Foundation (BMGF), and the World Bank’s Development Impact Evaluation (DIME).
The results showed that high-quality impact evaluations can be a tool for improving maternal and child health. Key findings were presented to health officials at the meeting, with a focus on their implications and practical value for the design of new policies and programs.
Impact Evaluations (IEs) use rigorous scientific methods to identify whether a policy or project has achieved the desired change that it set out to. This research method allows policymakers to take evidence-based decisions on how to scale-up a program, and can also allow changes to be made during the implementation of a policy or programme, based on evidence collected during implementation.
The Federal Ministry of Health (FMOH) has been working in partnership with the World Bank’s Development Impact Evaluation (DIME) team since 2007, to develop evidence about how Nigeria can best invest to improve the health of its population. Over the years, this collaboration has seen the implementation of several impact evaluations (IEs), dissemination events, and the building of a Community of Practice for those conducting impact evaluations in Nigeria.
At the event, five studies carried out in Nigeria using impact evaluations allowed researchers to show what changes the projects had brought to the lives of the people affected.
An evaluation of the SURE-P MCH project found that deploying midwives to underserved areas can improve the number of births that take place at the health facility, which is critical in reducing maternal mortality. However, increasing human resources and further supply-side investments such as providing more drugs and equipment did not improve the rate of antenatal care. This highlights the importance of not just focusing on providing staff and facilities, but also reducing the barriers that prevent women from attending and using antenatal services. More information on this study can be found here.
Another study showed that providing information, coaching, and mentoring to primary healthcare facility staff can improve the quality of care provided. However, providing information alone, without coaching and mentoring was not effective.
The effectiveness of delivering malaria-related primary health services in partnership with patent medicine vendors and community volunteers was evaluated in another study. The key findings were that the effectiveness of the partnerships was linked to the underlying quality of existing public primary healthcare facilities. In places where the quality of the existing public facility was high, the partnerships were more effective, but on average, no improvements were seen in malaria-related health outcomes.
In a study of MTV’s Shuga, results showed that delivering behaviour change communication through high-quality entertainment (edutainment) works to improve knowledge and behaviour relating to sexual and reproductive health, including reducing risky sex among youth. The use of mass media for behaviour change is, therefore, a promising low-cost, high-return strategy.
Finally, an evaluation of a programme that offered health information and malaria tests to agricultural workers showed that testing for malaria improved the productivity of health workers independent of test results. Simply providing tests resulted in an increase in productivity, even in those who tested positive for malaria. This suggests that there is economic value in simply knowing one’s health status.
The findings from these studies provide a basis for decision makers in the health sector in Nigeria to understand the tangible impact that a project makes on a community.
With the information from impact evaluations, supported by evidence-based research, policymakers can gain the confidence to make changes, communicate the potential impact of the changes and implement health programmes in their communities that will bring about lasting impact for all Nigerians.
We at DIME are grateful to the FMOH, the Bill & Melinda Gates Foundation, and to our many partners for the opportunity to contribute to this work. We look forward to further work together, to continue learning and to translate the lessons thus far into concrete policy actions that contribute to saving and improving lives.