Can Combining Performance-Based Financing With Equity Measures Result in Greater Equity in Utilization of Maternal Care Services? Evidence From Burkina Faso

Background

There is a growing concern that health inequalities related to social determinants of health are responsible for the slow progress witnessed in health and healthcare at global, regional and country levels, potentially jeopardizing opportunities to achieve the health related Sustainable Development Goals (SDGs). ¹ As countries embark on health financing and service delivery reforms, often targeting women and children first, monitoring health and healthcare inequalities remains an essential element of tracking progress towards the SDG 3 target of achieving universal health coverage by 2030, ensuring that no one is left behind. ² Monitoring is even more urgent and important in low- and middle-income countries (LMICs) because, despite the recent progress made in curbing maternal and child deaths, serious inequities in maternal and child health persist, especially in sub-Saharan Africa. ³ Exacerbating the situation, LMICs also face huge coverage gaps, health system inefficiencies, and insufficient quality of service delivery. ⁴

In recent years, much attention has been paid to performance-based financing (PBF) as a possible strategy to improve health system performance. PBF aims at reorienting health providers’ behavior towards provision of more and better quality care through the implementation of performance contracts that reward the attainment of predefined targets. ⁵ One vividly debated program design topic is a PBF programs’ potential to reinforce rather than to counteract existing inequities. ^6,7 Still, very few studies have looked at the equity impact of PBF on health services use and maternal care services in particular. While some studies suggest pro-least poor effects, ^8,9 others find evidence for the opposite ¹⁰ or distributional-neutral effects. ^11,12 Given the mixed evidence, an increasing number of authors advocate the introduction of PBF designs deliberately aimed at spreading program benefits more evenly across wealth groups. ^8,11,12 However, to date limited evidence is available. ¹³

Our study evaluates an “equity-conscious” PBF design, which was recently implemented in Burkina Faso and which was piloted with 3 different equity measures. We examined program effects on maternal health service utilization and defined equity as equal access given equal need, with access measured as reported service use and need, ¹⁴ measured in terms of a woman’s pregnancy status.

Methods

Study Setting

Burkina Faso is a landlocked West African country with a population of 18.6 million and a life expectancy of 60 years. Infant and under-five mortality rates stand respectively at 61 and 89 deaths per 1000 live births. An estimated 41.1 percent of the population live below the national poverty line of US$1.90 a day. Maternal mortality remains high at an estimated 371 per 100 000 live births. Multiple challenges related to maternal care persist, including serious inequities in access linked to sub-standard quality, low geographical accessibility, and financial barriers. ¹⁵

Prior to PBF, Burkina Faso undertook several health financing reforms to increase coverage and reduce inequities in access to and utilization of maternal health services, such as removal of user fees for antenatal care (ANC) services in 2002, and an 80% removal of user fees for delivery care in 2007, with a provision for full exemption of the ultra-poor. ¹⁶ Later in 2016, with the introduction of national free healthcare policy, known as the gratuité, the government removed all user fees for services delivered to children under the age of 5 years and to pregnant and lactating women. ¹⁷ As described below, the introduction of the national free healthcare policy induced the Ministry of Health to modify PBF prices, specifically to remove the equity measure additional payments (implemented in PBF2 and PBF3) for selected services targeted by both the national free healthcare policy and the PBF program. ¹⁸

The Intervention and the Study Design

Following an initial pilot in the districts of Titao, Leo, and Boulsa, starting in January 2014, Burkina Faso piloted PBF combined with different equity interventions in 12 districts distributed across 6 regions (Boucle du Mouhoun, Centre-Nord, Centre-Ouest, Nord, Sud-Ouest, Centre-Est) in which health facilities were rewarded by the Ministry of Health for achievement of defined health service indicators using a case-based payment system, adjusted for quality of care after verification. More details on the intervention design have been described elsewhere. ¹⁸ In brief, PBF was implemented according to 4 different models, 3 of which included an equity intervention targeting specifically the ultra-poor, as summarized in Table 1. The details of the ultra-poor selection process have been described elsewhere. ^19,20

Policy-makers expected the equity measures to induce increased utilization among the ultra-poor through 4 different pathways. First, they assumed that the targeting process would sensitize communities and particularly the targeted ultra-poor to the importance of health service utilization in case of need. Second, they assumed that the equity component would sensitize health workers to the importance of making specific efforts to facilitate health service utilization among the ultra-poor. Third, the removal of user fees for the targeted ultra-poor was assumed to reduce barriers to healthcare utilization among the ultra-poor. And finally, the elevated price levels for treating the targeted ultra-poor patients were assumed to enable and motivate health facilities to provide services to the ultra-poor free of charge.

To address our study primary objective of measuring the equity impact of PBF combined with equity measures compared to standard PBF alone, we inevitably needed to investigate the effect of PBF compared to status-quo service provision in the first place. To do so, we adopted a design that combined experimental with quasi-experimental elements. More specifically, we conducted a cluster-randomized trial nested within a pre-and post-test study with independent controls. Hereafter, we describe the different elements of our study in detail, referring to the quasi-experiment as study component 1 and to the cluster-randomized trial as study component 2. Figure provides a summary of the details of both study components including final sample sizes used in the analysis.

Close

Figure. Summary of Study Designs and Sampling Procedures. Abbreviation: PBF, performance-based financing.

Study Component 1

For the study component 1, six regions (Boucle du Mouhoun, Centre-Nord, Centre-Ouest, Nord, Sud-Ouest, Centre-Est) were identified non-randomly by the government and its development partners as intervention regions. Within each region, 2 districts were selected as intervention districts, ie, destined to receive PBF, and 2 districts (when not possible in a neighbouring region) as control, ie, continue with status quo service provision with no PBF. The intervention districts were purposely selected based on poor performance on selected maternal health indicators. ¹⁸ Control districts were selected to be as similar as possible (also in terms of performance on maternal health indicators) to intervention districts. This study component was set to allow a comparison between PBF districts (12) and districts (12) with status quo service provision without PBF. Since the intervention was assigned at district levels, districts effectively functioned as clusters for this study component.

Study Component 2

For the study component 2, ten out of 12 districts (2 with community-based health insurance (CBHI) where CBHI was pre-existing to allow implementation) were targeted by the Ministry of Health and development partners for randomization due to financial constraints (not enough funds to allow targeting across all 12 selected districts once calculations for targeting costs were made). In 8 out of 10 targeted districts in 4 regions (Centre-East, Centre-Nord, Sud-Ouest, Nord), clusters (primary healthcare facilities) were randomized to receive either PBF1 or PBF combined with either one of 2 equity measures, PBF2, and PBF3 so as to test the additional effect of combining standard PBF with equity measures as one way of reducing inequities in access to and utilization of health services.

Randomization took place within the framework of public randomization ceremonies in which concerned district and regions took turns in drawing primary healthcare facility names from a box containing all primary healthcare facility names in the 4 regions (Centre-East, Centre-Nord, Sud-Ouest, Nord) starting with the selection of pre-defined PBF model, and followed by the assignment of primary healthcare facilities in the order in which they were drawn from the box. ¹⁸ For example, in the 8 districts of the 4 regions (Centre-East, Centre-Nord, Sud-Ouest, Nord), this was done as follows: first facility: PBF1, second facility: PBF2; third facility: PBF3, fourth facility: PBF1, fifth facility: PBF2; and sixth facility: PBF3; etc. In these 8 districts of the 4 regions (Centre-East, Centre-Nord, Sud-Ouest, Nord) concerned by the three-arm randomization, this resulted into samples of 90 PBF1 facilities, 83 PBF2 facilities, and 84 PBF3 facilities. In the Boucle du Mouhoun region where 2 districts already implementing CBHI were targeted, 59 facilities were randomized to receive either PBF1 or PBF4 (following the same procedure outlined above), generating samples of 29 PBF1 and 30 PBF4 facilities. The 18 facilities which had been implementing CBHI prior to the launch of the study were excluded from randomization and hence from our study. Nevertheless, for ethical reasons, these facilities all implemented PBF in addition to CBHI. This study component was set to allow us to measure the benefit of combining PBF with an equity measure compared to implementing standard PBF on its own. In particular, we used this experimental component to measure the additional equity effects of PBF2, PBF3, and PBF4 compared to PBF1.

Sampling and Data Sources

For both study components, we used repeated cross-sectional household survey data collected at baseline from November 2013-March 2014 and at endline from April-June 2017. Sampling followed a three-stage cluster sampling procedure. First, for each primary healthcare facility included in the study (416 in intervention districts and 117 in the control districts — the number of facilities included in the study is larger for intervention compared to control districts since in intervention districts, we took a census of all facilities while in control districts we randomly selected one third of all facilities), we randomly selected one village. Second, within each village, we randomly selected 15 out of all households identified in each village where at least one woman was pregnant or had completed a pregnancy in the prior 24 months (inclusion criteria). Third, within a household, we interviewed all women of reproductive age (15-49 years), irrespective of whether they had a recent history of pregnancy.

The survey collected information on use of reproductive and maternal health services from women of reproductive age (15-49 years). Data on use of family planning were collected from all women of reproductive age regardless of marital status while data on use of maternal health services were collected only from women with a recent pregnancy. At baseline in study component 1, our sample comprised 9999 (7766 in intervention group, 2233 in control group) women of reproductive age (15-49 years) of whom 6568 (5074 in intervention group, 1494 in control group) had completed pregnancy 24 months prior to the survey. For the study component 2, at baseline, our sample comprised 6292 (1730 in PBF1, 1602 in PBF2, 1606 in PBF3 in the 4 regions [Centre-East, Centre-Nord, Sud-Ouest, Nord] and 521in PBF1 and 833 in PBF4 in Boucle du Mouhoun) women of reproductive age (15-49 years) of whom 4052 (1112 in PBF1, 970 in PBF2, 1026 in PBF3 in the 4 regions [Centre-East, Centre-Nord, Sud-Ouest, Nord] and 363 in PBF1 and 581 in PBF4 in Boucle du Mouhoun) had completed pregnancy 24 months prior to the survey. At endline, in study component 1, our sample comprised 11 010 (8432 in intervention group, 2578 in control group) women of reproductive age (15-49 years) of whom 6371 (4932 in intervention group, 1439 in control group) had completed pregnancy 24 months prior to the survey. For the study component 2, at endline, our sample comprised 6856 (1884 in PBF1, 1722 in PBF2, 1752 in PBF3 in the 4 regions [Centre-East, Centre-Nord, Sud-Ouest, Nord], and 570 in PBF1 and 928 in PBF4 in Boucle du Mouhoun) women of reproductive age (15-49 years) of whom 4065 (1099 in PBF1, 983 in PBF2, 1010 in PBF3 in the 4 regions [Centre-East, Centre-Nord, Sud-Ouest, Nord], and 368 in PBF1 and 605 in PBF4 in Boucle du Mouhoun) had completed pregnancy 24 months prior to the survey.

Variables and Their Measurement

Table 2 summarizes all outcomes and control variables. Our outcomes were selected to capture service coverage (defined as utilization given need, ie, pregnancy status) along the reproductive and maternal health service continuum and to reflect services which were incentivized by the PBF program, namely: ANC in the first trimester, at least 4 antenatal care (ANC4+) visits, at least 2 doses of tetanus toxoid vaccine (TTV2+), iron supplementation, HIV testing in pregnancy, facility-based delivery, at least 1 postnatal care (PNC1+) visit, at least 3 postnatal care (PNC3+) visits, and modern family planning methods (female sterilization, male sterilization, intrauterine device [IUD]/spiral, injectables/depoprovera, implants/norplant, male condom, female condom, diaphragm, foam/jelly). To improve the estimation precision, we included a number of control variables, which have the potential to explain the variation in outcome indicators from our previous work. ²¹ We relied on multiple correspondence analysis — run separately on baseline and endline samples —to generate a wealth index based on asset ownership and dwelling characteristics. ²² Given our specific research focus and the intervention’s intention to treat the ultra-poor ²³ and in line with prior literature, ⁸ we divided households in 2 wealth brackets corresponding to the Lowest 20% (ie, ultra-poor) and the rest — Upper 80%.

Data Analysis

Bivariate Analysis

First, we used t tests to assess systematic differences in the distribution of Outcome variables and Control variables across study arms for both study components.

Regression Analysis

Second, to assess the overall impact of PBF compared to status quo, we relied on study component 1 and used a difference-in-differences (DID) estimation approach, ²⁴ comparing intervention districts (irrespective of specific study arm) with control districts. We estimated a linear probability model, where we clustered standard errors at district level. In addition, for each outcome, we included village fixed effects and several individual-level covariates (equation 1):

(1) Ydvit=αv+β.Y17t+β.PBFd+λ.PBFd*Y17t+δ.Xit+εdvit

where Y_dvit outcome for individual i from villagev in district d at time t with t as (baseline, endline ); Y17_t is dummy variable representing endline; PBF_d is dummy variable denoting PBF exposure (1 = PBF, 0 = control); α_v is village fixed effects capturing time-invariant unobserved differences across villages. X_it is vector of individual-level covariates; and ε_dvit is error term. λ is the variable of interest (interaction term between PBF and endline) that gives the DID estimate for the effect of being located in a PBF district.

To determine overall PBF effects compared to status quo by socio-economic status group, we estimated regression model 1 by wealth bracket, following Lannes et al. ⁸

Third, to answer our key question on the equity impact of the PBF models integrating equity interventions, we relied exclusively on study component 2 of our study (10 districts) and also used DID to estimate a linear probability model as in equation 1, but with standard errors clustered at village level where randomization occurred. In this study component, equation 2 and equation 3 pertain, respectively, to the 8 districts in 4 regions (Centre-East, Centre-Nord, Sud-Ouest, Nord) where PBF1, PBF2, and PBF3 were randomized and to the 2 districts in Boucle du Mouhoun region where PBF1 and PBF4 were randomized:

(2) Yvit=αv+β.Y17t+β.PBF2v+λ2.PBF2v*Y17t+β.PBF3v+λ3.PBF3v*Y17t+δ.Xit+εvit (3) Yvit=αv+β.Y17t+β.PBF4v+λ4.PBF4v*Y17t+δ.Xit+εvit

where Y_vit is outcome for individual i from village v at time t with t as (baseline, endline ) in the intervention districts. λ₂ and λ₃ are variables of interest that give the DID estimates for the effects of being resident in PBF2 and PBF3 compared to PBF1, respectively, and λ₄ is the variable of interest that give the DID estimate for the effect of being resident in PBF4 compared to PBF1 in the Boucle du Mouhoun region.

Similarly to what we described earlier, to estimate specific effects by socio-economic status, we performed separate analyses by wealth bracket. ⁸

Furthermore, for study component 1, we performed several robustness checks to account for the small number of clusters (24 districts). We did so in light of the existing literature suggesting that: (1) a small number of clusters results in a higher likelihood of estimating downwards-biased standard errors, potentially leading to over rejection of the null hypothesis, ie, suggesting significant program impact while in reality there is none or very little impact ²⁵ ; and (2) bias arising from a small number of clusters is more acute in situations characterized by an imbalance in cluster sample sizes. ²⁶ Hence, to account for these 2 problems pertaining to our study component 1, we relied on the ‘wild bootstrap’ method for the related analyses. This method relies on a bootstrap t- procedure instead of bootstrapping the standard errors. ²⁵

We performed all analyses using Stata14 (Stata Corporation, Texas, USA).

Results

The results of our study are presented according to study components as follows:

Study Component 1

Bivariate Analysis

Tables 3 and 4 shows bivariate analysis of the characteristics of women of reproductive age (15-49 years) and coverage of maternal health services at baseline, respectively. At baseline, women in intervention and control districts were comparable on most demographic characteristics except age (Table 3).

In contrast, significant differences between PBF and control group existed in baseline values for a number of outcome variables: ANC4+ visits, HIV testing in pregnancy, facility-based delivery, PNC1+ visit and PNC3+ visits (Table 4).

Regression Analysis

Table 5 summarizes the results of the regression models pertaining to the overall impact of PBF compared to status quo, both for the entire sample and stratified by socio-economic group for the study component 1. We detected a positive effect of PBF on utilization of facility-based delivery [4.4 percentage points (pp) (P < .1)] and for PNC3+ visits: [6.6 pp (P < .1)]. This effect was primarily driven by an effect among the upper 80% of 5.5 pp (P < .05) and of 7.2 pp (P < .1) for facility-based delivery and PNC3+ visits, respectively. Among the poorest 20%, we detected an increase attributable to PBF for utilization of modern family planning methods of 7.6 pp (P < .1).

Table 6 summarizes the results of the robustness tests —using the “wild bootstrap” method. ²⁵ The results show that the estimates included in this study were all within the 95% confidence interval, and as such, there is no concern that the DID estimates in this study component are substantially biased due to the small number of clusters and/or to imbalances in cluster sample sizes. These results imply that we are not at risk of detecting an effect as significant when there was in fact no effect.

Study Component 2

Bivariate Analysis

Tables 7 and 8 presents bivariate analysis of the characteristics of women of reproductive age (15-49 years) and coverage of maternal health services, respectively at baseline in PBF2, PBF3 and PBF1 in the 4 regions (Centre-East, Centre-Nord, Sud-Ouest, Nord) and PBF4 and PBF1 in Boucle du Mouhoun region. At baseline, women across the 4 PBF arms were comparable except for marital status, distance to primary healthcare facilities, and age (Table 7).

The random allocation of facilities to the 4 PBF intervention arms resulted in uniform allocation for the majority of the outcome variables intervention arms, and utilization of most maternal care services increased over time with the exception of TTV2+ (Table 8).

Regression Analysis

Table 9 summarizes the results of the regression models aimed at estimating the additional benefit of PBF2, PBF3, and PBF4 compared to PBF1 for study component 2. Only PBF4 appeared to produce additional benefits, with significant positive effects on TTV2+ by 13.1pp (P < .05) and iron supplementation by 6.2 pp (P < .05) over and above PBF1. PBF2 performed worse than PBF1 in terms of its effect on utilization of TTV2+ by 6.8 pp (P < .1), while PBF3 had negative additional effects on facility-based delivery by 3.8 pp (P < .1) and PNC1+ visit by 6.7 pp (P < .1) respectively.

Table 10 represents the core of our analysis, as it summarizes the results of the regression models aimed at estimating the additional benefit of PBF2, PBF3, and PBF4 compared to PBF1 by socio-economic subgroup for study component 2. Similar to the overall findings presented in Table 9, the equity measures that accompanied the implementation of PBF did not result in any additional benefit for the poorest 20%, but rather the opposite on certain indicators. PBF2 and PBF3 decreased utilization of facility-based delivery by 11.7 pp (P < .05) and 11.8 pp (P < .05), respectively among the poorest 20%. In addition, PBF3 decreased utilization of iron supplementation by 7.4 pp (P < .05) and modern family planning methods by 12.7 pp (P < .05) among the poorest 20%, while PBF4 decreased utilization of PNC1+ visit by 24.1 pp (P < .05) among the poorest 20%. The overall positive additional effect of PBF4 on TTV2+ and iron supplementation coverage (seen in Table 9) was present only among the upper 80%, with an increase of 13.7 pp (P < .05) and 7.6 pp (P < .05) respectively, but not among the poorest 20%.

Discussion

This study makes a unique contribution to the literature by combining experimental and quasi-experimental elements to investigate not only the overall impact of PBF on maternal health service coverage, but specifically the role of combining equity interventions with standard PBF to reduce existing inequities.

Unfortunately, the results do not correspond to what policy-makers and their development partners had intended to achieve when designing the intervention and even yield opposite results. It is unfortunate that no data on costs of any interventions (PBF, equity measures, gratuité) are available which would have allowed policy-makers to put the results into a better perspective. Nevertheless, appraising findings across our multiple strains of analysis, it appears that while PBF produced modest changes compared to status quo, the implementation of equity interventions did not generate additional benefits compared to PBF alone, neither for women in general nor for the poorest women specifically. In a few selected instances, PBF combined with equity interventions even resulted in worse outcomes than PBF alone. This observation is aligned with some published evidence. For instance, in Cambodia, coupling PBF with maternity vouchers to cover user fees for the poor was also observed not to improve service utilization for the poor. ²⁷ Two studies from Rwanda, where PBF was coupled with CBHI in the analysis, showed mixed results. One study found that PBF yielded no equity effects, ¹² while the other study detected pro- poor effects for utilization of facility-based deliveries but negative equity effect on use of modern family planning methods among the poor. ⁸

Before we attempt to uncover reasons for why PBF did not attain the intended equity effects, we shall briefly comment on our finding regarding facility-based delivery. While an absolute change of 4 percentage points at a significance level of 10% may appear to be negligible, it is in fact remarkable considering the extremely high baseline utilization values, approaching 90%. In addition, we ought to consider that study component 1 was largely underpowered due to the constraints imposed by the small number of clusters, equivalent to 24 districts. Since the least-poor drove changes in utilization of facility-based delivery, it is plausible to assume that PBF might have produced positive changes in quality of service delivery necessary to encourage further utilization among those with means to be receptive to quality improvements. Further research into the impact of PBF on quality of service delivery is needed to verify this hypothesis. The positive change observed on PNC is less striking, since baseline utilization values departed from relatively low levels. Nevertheless, this change is highly relevant given that health systems currently struggle to increase use of PNC services. ²⁸ The general decline in coverage of TTV2+ from baseline to endline in both PBF and control catchment areas may be due to the fact that by the time the endline data were collected, most women in the catchment areas had received the 5 doses stipulated by government policy and were therefore not eligible to receive additional vaccination. This issue arose as the question in the survey was set to capture new vaccinations rather than overall vaccination coverage.

Understanding the lack of additional benefit produced by the more complex PBF models integrating equity interventions compared to standard PBF requires a closer inspection of the study context. Endline data collection took place between April and June 2017, approximately one year after the launch of the national free healthcare policy targeting women and children. ¹⁷ Given a pregnancy recall period of 24 months, this means that by the time we collected endline data, only a portion of women in our study area had been exempted from payment of user fees for all maternal care services except modern family planning methods, irrespective of whether they lived in the catchment area of a standard PBF facility (PBF1) or in areas with an additional targeting and subsidization of the ultra-poor (PBF2, PBF3) or a CBHI (PBF4) model.

It could be argued that following the launch of the national free healthcare policy, health service use for the specific indicators included in our study among the ultra-poor might have caught up so fast in PBF1 areas due to the removal of the financial barrier to make it impossible for us to detect any effect of the PBF equity interventions which might have been there prior to June 2016. Our analysis, however, clearly indicates that saturation (ie, utilization rates of 100%) was not reached for any of the targeted indicators. In addition, we note that our effect estimation is by no means invalidated by the implementation of the national free healthcare policy for maternal health, since pre- and post-test designs with independent controls and relying on a DID analytical approach are not compromised by presence of group-invariant factors, such as policies launched across all districts in the country simultaneously. ²⁹ As such, if the national free healthcare policy did bear any effect on service utilization (which we do not know because this is beyond the scope of this study), it is likely to have done so in all PBF models and control districts, not affecting in any way our ability to detect differences between PBF and control districts and across PBF models.

Nevertheless, we need to acknowledge that the introduction of the national free healthcare policy induced policy-makers to adjust the implementation of the equity measures in PBF2 and PBF3. Specifically, qualitative interviews with key stakeholders revealed that following the launch of the national free healthcare policy, the Ministry of Health removed additional compensation to healthcare facilities in PBF2 and PBF3 for all those services which were included in the free healthcare policy benefit package. This means that effectively, by the time we collected endline data, PBF2 and PBF3 were equivalent to PBF1 in terms of incentives related to all maternal care services except modern family planning. This could well have demotivated health providers from seeking innovative strategies for reaching out to provide the poor with the needed services. ³⁰ It ought to be noted, however, that the introduction of the national free healthcare policy only touched the assumed financial mechanisms of the equity components, while effects of the sensitization mechanisms activated by the targeting exercise should have remained constant in PBF2 and PBF3 facility catchment areas only, but not in PBF1 facility catchment areas. As such, the introduction of the free national healthcare policy could have diluted, but not fully removed, the effect of the equity measures, had there been one in the first place.

The fact that we detected a negative effect of PBF2, PBF3, and PBF4 compared to PBF1 on selected indicators, especially when considering the stratified analysis looking only at the poorest, is worrisome. However, this appears to corroborate existing evidence pointing at the presence of unintended consequences related to the implementation of the community-based targeting and related subsidized program ³⁰ and at general challenges related to the implementation of the overall PBF program in the country. ^31,32 Appraising our current findings in light of existing literature suggests that combining PBF and equity interventions into a single intervention might have resulted in a level of complexity not easily manageable for front-line healthcare providers, ultimately leading to effects contrary to the ones that had been anticipated. For example, evidence shows that health providers introduced ceilings to services offered to the ultra-poor as a way of adapting to the complexity of the PBF interventions and to the long delays in receiving incentive payments, which created financial difficulties for health workers. ³⁰

Albeit worrisome, the results of our analysis are not per se surprising as they align with PBF evidence from other settings as well as with prior research assessing the impact of earlier targeted exemption policies in Burkina Faso. For instance, the national obstetric care policy implemented from 2007 to 2016 failed to reach the poorest women effectively. ³³ A recently published study indicated that lack of fidelity in implementing exemption policies may be due to providers’ lack of adequate knowledge in the first place. ³⁴ This suggests a need to educate providers on the purpose and procedures of a given policy to transform them into real agents of change, since poor communities are not sufficiently empowered to overcome all relevant barriers to access in response to a single targeting mechanism. Further qualitative inquiry is needed to unravel if and to what extent providers’ understanding of the targeted exemptions represented a barrier to the effective implementation of equity interventions in Burkina Faso.

In addition, it is possible that mere removal of user fees through targeting was insufficient to enable very poor people to seek care. Prior evidence from the region and the country specifically clearly points at the presence of important non-financial barriers to access. ^33,35 For example, inequities existed in facility-based delivery due to distance to catchment primary health facility, literacy, parity and religion. ²¹ There were also inequities in utilization rates for ANC4+ visits due to distance to catchment primary health facility, literacy, parity, religion and marital status and for PNC1+ visit due to distance to catchment primary health facility, age and religion. ²¹ Since the PBF program did not address these other sources of inequity, other than household wealth, arguably they still constituted barriers to uptake of essential maternal health services by the poorest women. Still, further research is needed to unravel why in some settings equity interventions are not effective in narrowing equity gaps while in others, such as Tanzania, ¹⁰ combining PBF and targeted exemptions resulted in greater service use among the poor in public health facilities. This is in line with Renmans et al, who have observed that although PBF has received increasing attention, a lot remains unknown about the exact mechanisms triggered by PBF arrangements. ³⁶ As such, they have called for more research to examine the exact mechanisms through which not only incentives, but also ancillary components operate. Such knowledge is necessary to understand and appreciate the effectiveness, desirability and appropriateness of PBF as a possible tool towards health systems strengthening in LMICs.

Methodological Considerations

Our study is not without limitations. First, since the intervention took place within a real-life setting, we cannot rule-out that other interventions with similar objectives took place alongside PBF, especially in control districts. Hence, we cannot estimate the extent to which our comparator really reflects status quo utilization rates. Second, we need to acknowledge the fact that women identified in our study as the poorest do not exactly match the ones identified by the community-based targeting procedure of the program as such. Hence, the reader ought to be aware that our findings illustrate the impact on the lowest quintile in general and not on targeted individuals specifically. Parallel research efforts based on a different dataset are ongoing to look at the impact of the equity interventions specifically on targeted individuals. Third, the power to detect impact in study component 1 was limited by the relatively low number of clusters. This limitation was noted well in advance, when the overall PBF impact evaluation study was being designed, but financial and policy challenges made it impossible to increase the number of clusters, and hence it was agreed amongst all key stakeholders to live with this limitation. Fourth, the purposive selection of the districts represents a potential threat to external validity, more specifically to the generalizability of the results emerging from study component 1. Precisely, the purposive selection of the districts does not allow us to make inferences about the possible effects of PBF in districts with higher baseline values, hence we need to exercise caution in generalizing the results of this study to other contexts. This purposive selection, however, does not represent a threat in terms of the internal validity of the DID analysis, since it does not violate the basic assumptions of the DID model. ^29,37 Fifth, as discussed extensively earlier, the modifications which were operated to the PBF design following the introduction of the national free healthcare policy could have potentially contributed to diluting, but not eliminating, the effect of PBF, had there been one in the first place.

Conclusion

PBF is being implemented in LMICs and sub-Saharan Africa in particular as a response to weak health systems performance. Although rapidly growing, evidence regarding its effectiveness is still very mixed. Evidence regarding PBF equity impacts on use of health services and maternal health is particularly scarce and, when available, it is mixed, in some cases conflicting. Our results indicate that even well-designed PBF interventions which integrate explicit equity components are not sufficient to overcome inequities in health service use. As such, our results confirm the need for additional interventions reaching beyond the financial realm to ensure access to care by the ultra-poor. In addition, our findings suggest that changing policy environments inevitably affect the way an intervention, in this case PBF combined with equity measures, is carried out and hence should be explicitly acknowledged when appraising effects.

Lastly, we would like to reiterate the importance of carefully monitoring and measuring the equity impact of interventions targeted at improving access and quality of service delivery as an integral element of SDG 3. The experience of the PBF program in Burkina Faso provides a clear illustration of how even well-intended and accurately designed interventions may fail to achieve their objective due to a variety of contextual elements that shape implementation in unattended manners. As such, greater attention to context and competing policies ought to be paid in designing strategies that aim at building synergies between supply and demand to overcome existing inequities.

Acknowledgements

This work was supported by the World Bank through the Health Results Innovation Trust Fund (HRITF). We thank the Programme d’Appui au Développement Sanitaire (PADS) for financial support for data collection given to Centre Muraz. We also thank Aurélia Souares, Nobila Ouedraogo, Hien Herve, and Herman Badolo for successfully coordinating baseline and endline data collection, respectively and all Muraz staff and interviewers for the fieldwork.

Ethical issues

The ethical clearance for the study was obtained from the Ethical Committee of Heidelberg University (S-272/2013) and from the National Ethics Committee in Burkina Faso (N° 2013-7-066 and N° 2015-5-071).

Competing interests

This work was supported by the World Bank through HRITF, which was administered by the Medical Faculty of the University of Heidelberg. The World Bank was engaged in the overall design of the intervention and the impact evaluation, but had no role in data collection, data management, data analysis and interpretation, preparation, review and approval of the manuscript. MDA was the Principal Investigator of the IE, but received no direct compensation from the World Bank. TM is a doctoral student at the University of Heidelberg, who was not part of the IE and contributed his own time towards conception of the study, data analysis and interpretation, and drafting of the manuscript. JL, MS, and SB were fully or partially funded by the HRITF grant to the University and worked on the IE (data collection, management, and reporting to the World Bank), but received no direct payment by the Bank nor any compensation for manuscript preparation (which occurred outside the framework of the contract with the World Bank). SS and HH are employees of Centre MURAZ, but were not directly funded by the HRITF grant allocated to Centre MURAZ for data collection. SH is a World Bank employee based in Yaoundé, Cameroon and contributed to the data collection and management in the context of the IE, but participated in writing of this paper independently of his professional engagement. The views reported in this paper represent the views of the authors exclusively and not those of the funding agency.

Authors’ contributions

TM, MDA, and JL conceived the study. JL, MDA, SB, SMAS, SH, and HH contributed to survey design and data collection. TM undertook data analysis with support from JL, MDA, and MH. All authors contributed to the interpretation of the results. TM drafted the manuscript with contributions from all authors.

Authors’ affiliations

¹Heidelberg Institute of Global Health, Medical Faculty and University Hospital, Heidelberg University, Heidelberg, Germany. ²Department of Global Health and Development, London School of Hygiene & Tropical Medicine, London, UK. ³The World Bank, Yaoundé, Cameroon. ⁴Centre MURAZ, Bobo-Dioulasso, Burkina Faso. ⁵UFR/ST, Université Nazi Boni, Bobo-Dioulasso, Burkina Faso.

References

1. Rasanathan K. 10 years after the Commission on Social Determinants of Health: social injustice is still killing on a grand scale. Lancet 2018; 392(10154):1176-1177. doi: 10.1016/s0140-6736(18)32069-5 [Crossref] [ Google Scholar]
2. Hosseinpoor AR, Bergen N, Magar V. Monitoring inequality: an emerging priority for health post-2015. Bull World Health Organ 2015; 93(9):591-591A. doi: 10.2471/blt.15.162081 [Crossref] [ Google Scholar]
3. UNICEF. UNICEF data: Monitoring the Situation of Children and Women-Maternal Mortality. https://data.unicef.org/topic/maternal-health/maternal-mortality/. Accessed July 14, 2020. Published 2019.
4. Moucheraud C, Owen H, Singh NS. Countdown to 2015 country case studies: what have we learned about processes and progress towards MDGs 4 and 5?. BMC Public Health 2016; 16 Suppl 2(Suppl 2):794. doi: 10.1186/s12889-016-3401-6 [Crossref] [ Google Scholar]
5. Renmans D, Holvoet N, Criel B, Meessen B. Performance-based financing: the same is different. Health Policy Plan 2017; 32(6):860-868. doi: 10.1093/heapol/czx030 [Crossref] [ Google Scholar]
6. Ireland M, Paul E, Dujardin B. Can performance-based financing be used to reform health systems in developing countries?. Bull World Health Organ 2011; 89(9):695-698. doi: 10.2471/blt.11.087379 [Crossref] [ Google Scholar]
7. Friedman J, Scheffler RM. Pay for performance in health systems: theory, evidence and case studies. In: Scheffler RM, ed. World Scientific Handbook of Global Health Economics and Public Policy. Singapore: World Scientific; 2016:295-332. World Scientific Series in Global Health Economics and Public Policy. 10.1142/9789813140530_007
8. Lannes L, Meessen B, Soucat A, Basinga P. Can performance-based financing help reaching the poor with maternal and child health services? the experience of rural Rwanda. Int J Health Plann Manage 2016; 31(3):309-348. doi: 10.1002/hpm.2297 [Crossref] [ Google Scholar]
9. Bonfrer I, Van de Poel E, Van Doorslaer E. The effects of performance incentives on the utilization and quality of maternal and child care in Burundi. Soc Sci Med 2014; 123:96-104. doi: 10.1016/j.socscimed.2014.11.004 [Crossref] [ Google Scholar]
10. Binyaruka P, Robberstad B, Torsvik G, Borghi J. Who benefits from increased service utilisation? examining the distributional effects of payment for performance in Tanzania. Int J Equity Health 2018; 17(1):14. doi: 10.1186/s12939-018-0728-x [Crossref] [ Google Scholar]
11. Engineer CY, Dale E, Agarwal A. Effectiveness of a pay-for-performance intervention to improve maternal and child health services in Afghanistan: a cluster-randomized trial. Int J Epidemiol 2016; 45(2):451-459. doi: 10.1093/ije/dyv362 [Crossref] [ Google Scholar]
12. Priedeman Skiles M, Curtis SL, Basinga P, Angeles G. An equity analysis of performance-based financing in Rwanda: are services reaching the poorest women?. Health Policy Plan 2013; 28(8):825-837. doi: 10.1093/heapol/czs122 [Crossref] [ Google Scholar]
13. Ridde V, Gautier L, Turcotte-Tremblay AM, Sieleunou I, Paul E. Performance-based financing in Africa: time to test measures for equity. Int J Health Serv 2018; 48(3):549-561. doi: 10.1177/0020731418779508 [Crossref] [ Google Scholar]
14. Braveman P, Gruskin S. Defining equity in health. J Epidemiol Community Health 2003; 57(4):254-258. doi: 10.1136/jech.57.4.254 [Crossref] [ Google Scholar]
15. Atchessi N, Ridde V, Zunzunegui MV. User fees exemptions alone are not enough to increase indigent use of healthcare services. Health Policy Plan 2016; 31(5):674-681. doi: 10.1093/heapol/czv135 [Crossref] [ Google Scholar]
16. Belaid L, Ridde V. An implementation evaluation of a policy aiming to improve financial access to maternal health care in Djibo district, Burkina Faso. BMC Pregnancy Childbirth 2012; 12:143. doi: 10.1186/1471-2393-12-143 [Crossref] [ Google Scholar]
17. Ministère de la Santé du Burkina Faso. Santé au Burkina Faso: Les mesures de gratuité des enfants de moins cinq ans et des femmes enceintes lances. https://reliefweb.int/report/burkina-faso/sant-au-burkina-faso-les-mesures-de-gratuit-des-enfants-de-moins-cinq-ans-et-des. Accessed July 14, 2020. Published 2016.
18. De Allegri M, Lohmann J, Souares A. Responding to policy makers’ evaluation needs: combining experimental and quasi-experimental approaches to estimate the impact of performance based financing in Burkina Faso. BMC Health Serv Res 2019; 19(1):733. doi: 10.1186/s12913-019-4558-3 [Crossref] [ Google Scholar]
19. Beaugé Y, Koulidiati JL, Ridde V, Robyn PJ, De Allegri M. How much does community-based targeting of the ultra-poor in the health sector cost? Novel evidence from Burkina Faso. Health Econ Rev 2018; 8(1):19. doi: 10.1186/s13561-018-0205-7 [Crossref] [ Google Scholar]
20. Ridde V, Yaogo M, Kafando Y. A community-based targeting approach to exempt the worst-off from user fees in Burkina Faso. J Epidemiol Community Health 2010; 64(1):10-15. doi: 10.1136/jech.2008.086793 [Crossref] [ Google Scholar]
21. Mwase T, Brenner S, Mazalale J. Inequities and their determinants in coverage of maternal health services in Burkina Faso. Int J Equity Health 2018; 17(1):58. doi: 10.1186/s12939-018-0770-8 [Crossref] [ Google Scholar]
22. Blasius J, Greenacre M. . Multiple Correspondence Analysis and Related Methods in Practice 2006.
23. Ridde V, Turcotte-Tremblay AM, Souares A. Protocol for the process evaluation of interventions combining performance-based financing with health equity in Burkina Faso. Implement Sci 2014; 9:149. doi: 10.1186/s13012-014-0149-1 [Crossref] [ Google Scholar]
24. Angrist JD, Pischke JS. Mostly Harmless Econometrics: An Empiricist’s Companion. New Jersey: Princeton University Press; 2009.
25. Cameron AC, Gelbach JB, Miller DL. Bootstrap-based improvements for inference with clustered errors. Rev Econ Stat 2008; 90(3):414-427. doi: 10.1162/rest.90.3.414 [Crossref] [ Google Scholar]
26. Imbens GW, Kolesar M. Robust standard errors in small samples: some practical advice. Rev Econ Stat 2016; 98(4):701-712. [ Google Scholar]
27. Van de Poel E, Flores G, Ir P, O’Donnell O. Impact of performance-based financing in a low-resource setting: a decade of experience in Cambodia. Health Econ 2016; 25(6):688-705. doi: 10.1002/hec.3219 [Crossref] [ Google Scholar]
28. Lawn JE, Blencowe H, Oza S. Every Newborn: progress, priorities, and potential beyond survival. Lancet 2014; 384(9938):189-205. doi: 10.1016/s0140-6736(14)60496-7 [Crossref] [ Google Scholar]
29. Wing C, Simon K, Bello-Gomez RA. Designing difference in difference studies: best practices for public health policy research. Annu Rev Public Health 2018; 39:453-469. doi: 10.1146/annurev-publhealth-040617-013507 [Crossref] [ Google Scholar]
30. Turcotte-Tremblay AM, De Allegri M, Gali-Gali IA, Ridde V. The unintended consequences of combining equity measures with performance-based financing in Burkina Faso. Int J Equity Health 2018; 17(1):109. doi: 10.1186/s12939-018-0780-6 [Crossref] [ Google Scholar]
31. Bodson O, Barro A, Turcotte-Tremblay AM, Zanté N, Somé PA, Ridde V. A study on the implementation fidelity of the performance-based financing policy in Burkina Faso after 12 months. Arch Public Health 2018; 76:4. doi: 10.1186/s13690-017-0250-4 [Crossref] [ Google Scholar]
32. Ridde V, Yaogo M, Zongo S, Somé PA, Turcotte-Tremblay AM. Twelve months of implementation of health care performance-based financing in Burkina Faso: a qualitative multiple case study. Int J Health Plann Manage 2018; 33(1):e153-e167. doi: 10.1002/hpm.2439 [Crossref] [ Google Scholar]
33. De Allegri M, Tiendrebéogo J, Müller O, Yé M, Jahn A, Ridde V. Understanding home delivery in a context of user fee reduction: a cross-sectional mixed methods study in rural Burkina Faso. BMC Pregnancy Childbirth 2015; 15:330. doi: 10.1186/s12884-015-0764-0 [Crossref] [ Google Scholar]
34. Ridde V, Leppert G, Hien H, Robyn PJ, De Allegri M. Street-level workers’ inadequate knowledge and application of exemption policies in Burkina Faso jeopardize the achievement of universal health coverage: evidence from a cross-sectional survey. Int J Equity Health 2018; 17(1):5. doi: 10.1186/s12939-017-0717-5 [Crossref] [ Google Scholar]
35. Gabrysch S, Campbell OM. Still too far to walk: literature review of the determinants of delivery service use. BMC Pregnancy Childbirth 2009; 9:34. doi: 10.1186/1471-2393-9-34 [Crossref] [ Google Scholar]
36. Renmans D, Holvoet N, Orach CG, Criel B. Opening the ‘black box’ of performance-based financing in low- and lower middle-income countries: a review of the literature. Health Policy Plan 2016; 31(9):1297-1309. doi: 10.1093/heapol/czw045 [Crossref] [ Google Scholar]
37. Meyer BD. Natural and quasi-experiments in economics. J Bus Econ Stat 1995; 13(2):151-161. doi: 10.1080/07350015.1995.10524589 [Crossref] [ Google Scholar]