Improving Performance in Complex Surroundings: A Mixed Methods Evaluation of Two Hospital Strategies in the Netherlands

Background

The dominant position of hospitals in healthcare systems is challenged as care can increasingly be provided outside these institutions.¹ Hospitals are complex organizations that house a variety of models of care within a single organisation.² As financial pressure increases, policy-makers seek fundamental changes in hospital service delivery, aiming for appropriate care since inefficient services may lead to low value care, reducing both quality and financial sustainability.³

Hospital-initiated strategies that aim to improve quality and reduce costs are attractive and thus sought-after. Examples of such strategies are total quality management, lean, six sigma, and value-based healthcare. Previous case-based research demonstrated that improving quality of care may indeed reduce costs.^4-10 Although a range of case studies was conducted, evidence on real-world effectiveness across the entire hospital is relatively limited. Contextual factors, such as organisational culture, may account for mixed results across varied settings.^5,6,10 These studies may be subject to bias, since they are relatively scarce, and predominantly report positive effects, and are often conducted by authors affiliated with the institution.¹⁰

Moreover, insights in possible interactions between separate contextual factors are underdeveloped.¹¹ Traditional case study evaluations are mostly linear (eg, time series analysis) and have limited generalisability across varying context and settings. Quality improvements are often “facilitated evolutions.”¹² Earlier work shows a lack of integrated improvement research that combine evaluations of costs, quality of care and implementation.⁵ This study aims to add to current knowledge through the use of a multi-perspective analysis on complex hospital strategies, defined by local dynamic context and a nonlinear evolution. Such approaches to evaluations of complex adaptive systems have proven their value in the context of healthcare.^13-15

Two hospitals in the Netherlands have committed to such strategies and have taken explicit steps towards financial sustainability through quality improvement. Fundamental elements of their strategies were (1) hospital and payer alignment, (2) engagement of physicians, and (3) an orientation toward partners in their catchment area.^16,17 Early self-reported results indicated substantial improvements.^16,18 However, it is unknown how quality improvement processes were actually undertaken, which factors contributed to their outcomes, and how the effects translate into quality and costs at the hospital level.¹⁹ Approximately five years after the initiation of their reforms, we examined the extent to which both hospitals reached these goals. Our primary research question was “How can institution-wide improvement programmes contribute to the aim of reducing costs through quality improvement?” Multiple sources of data, both quantitative and qualitative, were used to examine contexts, mechanisms, and outcomes, following the Medical Research Council’s framework for complex interventions.^20,21

Institutional Context

Bernhoven is a mid-sized (approximately 380 beds) general hospital situated in the southeast of the Netherlands. The Beatrix hospital, part of Rivas care group, is a small (approximately 220 beds) general hospital in the southwest of the Netherlands. Both hospitals developed their vision for organisational change inspired by a consultancy report “Quality as remedy.”¹⁷ Both hospitals faced an urgent need for change. Increasing competition between providers and nearby larger (tertiary) centres threatened their future as smaller-sized general hospitals. In the case of Bernhoven, the funding of a new hospital building added to fiscal pressures. In the case of the Beatrix hospital, a planned merger with a large nearby hospital was not approved by the competition authorities.

Methods

Study Design

We conducted a convergent parallel mixed-methods study to evaluate contexts, mechanisms and outcomes of both hospital strategies. This enabled collection of multiple data sources, both quantitative and qualitative, with iterative cycles of validation and confirmation.²² This design complies with the Medical Research Council’s framework for the evaluation of complex interventions.^20,21 The evaluation was conducted between April 2019 and May 2020.

Data Collection

Four streams of data were collected. The first stream was collected with the aim of assessing implementation mechanisms. Specific quantitative outcomes were collected about quality of care (second stream), financial data (third stream), and additional analyses on care volumes (fourth stream). Two interactive sessions with representatives of both hospitals, insurers, the patients federation, and the Ministry of Health were used to report progress of the evaluation. These sessions were used to validate our findings and identify omissions in the evaluation.

First, we collected qualitative data through semi-structured interviews. The topic guide was constructed according the Consolidated Framework for Implementation Research.²³ This framework is suitable for evaluating the implementation process, inner and outer contexts, relevant stakeholders, and outcomes (Supplementary file 1). This framework was used as an extension to the Medical Research Council’s framework, as it consists of a more pragmatic structure for evaluations of complex interventions.²⁴ Second, we collected routinely measured hospital quality indicators, which are publicly available through the Health and Youth Care Inspectorate and the National Health Care Institute in the Netherlands.^25,26 These quality indicators are typically measured per specialty or disease group and are classified according to the Donabedian framework, which uses structure, process and outcome indicators to assess quality.²⁷ The number of indicators varies over time and ranges between a maximum of 1932 in 2012 and a minimum of 722 (2018), with the majority being structural indicators on quality of care (Table 1). The National Health Care Institute collects quality indicators and these indicator sets are adjusted and updated annually, resulting in variation across years. In addition, the National Healthcare Map (in Dutch: Zorgkaart Nederland) collects patient experiences from 2008 to 2018 with patients grading hospitals or caregivers on a voluntary basis.²⁸ The third data stream was drawn from annual reports and financial statements for both hospitals (2007−2020), which are publicly accessible in the Netherlands.²⁹ The main financial indicators that we used for our analysis were hospital revenue, expenses, solvency ratios, and gross and net margins. Trendlines for both intervention hospitals were compared to the (sector) trend of all hospitals in the Netherlands. The final stream of data collection consisted of several additional existing data sources, which were both published and unpublished data of trends in (specific) volumes and quality-improvement projects in both hospitals. Volume trends were analysed by the Netherlands Bureau for Economic Policy Analysis, using quantitative data.³⁰ Additional (unpublished) data on individual improvement projects were used as complementary material.

Recruitment Process and Sampling Strategy

The board of directors of both hospitals set up contacts with the strategy programme managers. Key participants were identified by the programme managers (which were interviewed as well) of both hospitals and through purposive sampling in interviews. Our aim was to conduct interviews across the full organization, targeting a variety of medical departments and support staff. External stakeholders (ie, general practitioner [GPs], healthcare insurers, and consultants) were contacted to include multiple perspectives on the transformation programmes. Potential participants were approached by email and a reminder was sent after two weeks. All participants provided written informed consent for this study. We interviewed 62 key stakeholders: medical doctors (n = 19), nurses (n = 5), hospital managers (n = 15), GPs (n = 7), primary care managers (n = 3), external consultants (n = 4), patients (n = 9), and insurer managers (n = 5). Of these stakeholders, five held dual functions (eg, medical doctors and managers). Interviews were held between May 2019 and April 2020. Interviews were conducted concurrently and iteratively to inform subsequent interviews. Supplementary file 2 provides an overview of all the interviewed stakeholders.

Data Analysis

Qualitative interviews were transcribed ad verbatim and analysed thematically in ATLAS.ti (version 8.4.20). The first ten interviews were analysed independently by three researchers to achieve consensus on the coding list, enhancing inter-researcher reliability. After agreement was reached, a single researcher analysed the remaining transcripts using the item list. Data were coded deductively using themes that describe features influencing complex implementation processes, identified in literature: organizational culture, networks and communication, leadership, resources (financial resources, staffing and workload, time, education and training), evaluation, monitoring, and feedback, and champion.¹¹ These themes were used for the identification of contextual factors. Interrelationships between different themes demonstrate overarching mechanisms that “can act as enablers in one implementation setting but barriers in others,” for example the effect of leadership on culture, communication and resources.¹¹ The identified mechanisms in this study follow evident interrelationships between themes. Finally, interviewing of respondents also revealed (perceived) qualitative outcomes, which were used as supplementary data to the quantitative findings (eg, patient satisfaction, perceived quality).

To evaluate the actual effects on quality of care, composite indicators were constructed following the standard-normal transformation methodology.^25,31,32 Each quality indicator was transformed into a z-score expressing deviation from the population mean:

z−scorei,j=Xi,j−X¯jσi,j

where x_i,j is the value of indicator j for hospital I, and σ is the standard deviation. The composite indicator is an unweighted sum of z-scores, reflecting the average deviation from the mean over indicator set J (ie, no differentiation is made in relative importance of individual indicators). A composite indicator was constructed for all routinely measured structural quality indicators (maximum of n = 1826 in 2012, minimum of n = 579 in 2018), process indicators (minimum of n = 31 in 2010, maximum of n = 114 in 2016), and outcome indicators (minimum of n = 14 in 2010, maximum of n = 56 in 2016) (See Table 1). As the content and number of indicators differ over time, constructing a benchmark z-score allows comparisons of the average relative quality performance over time. Data from all Dutch hospitals were used to calculate z-scores. The mean composite indicator scores of other hospitals in the Netherlands are calculated as well to compare trendlines between the intervention hospitals, and the hospital sector. The score therefore represents the relative performance of a hospital versus all other hospitals. An analytic narrative was constructed in the third data stream using data from annual financial reports, interview data, and external data from desk research. Both hospitals were offset against all Dutch hospitals. As the implementation programmes started in 2014 and 2015, 2012 was chosen as a base year to offset any anticipation effects. The median was used to correct for skewed data.

Difference-in-differences (DID) models were estimated to compare means and trends in the pre- and post-intervention period.³³ First, we tested whether mean total expenditures differed after intervention year 2015 in the intervention hospitals compared to control hospitals, after correcting for pre-intervention differences.^34-37 Next, we tested whether growth rates in total expenditures differed after 2015, correcting for pre-intervention differences in levels and rates of growth between intervention and control hospitals. We repeated the analysis for the part of total expenditures paid by insurers, solvency ratios, profit margins and quality indicators. Quality indicators were analysed separately for structure, process, and outcome quality. Only general hospitals were included in the DID models.

Results

The strategies of both hospitals were shaped into a complex multiyear implementation programme, consisting of different elements that cover the entire organisation and its stakeholder alignments. These are summarised in Table 2.

Six main configurations demonstrate how the strategy and implementation impacted quality and costs in both hospitals. Qualitative data identified and prioritized these six mechanisms. Primarily, both hospitals aimed to improve quality through improvement projects and through substitution of care to primary care settings (mechanisms 1, 2, and 3). Second, multiyear contracts, scaling down of internal costs, and altered remuneration of physicians were required to guarantee financial security for physicians and the hospital while volume of care declined as a result of the improvement projects (mechanisms 4, 5, and 6). Both hospital strategies were evolving processes: their main goals — improving quality and reducing costs — were not explicitly connected to clear targets that could allow for incremental changes to the initial strategy.

“The change process was complex: rather than implementing a clear intervention with clear goals, we tried to establish a mutual sense of direction. The direction was improved care for patients, financially sustainable for society and we respect each other’s roles” (i29, external consultant).

Quality-Improvement Projects Spur Change and Commitment

Context

At the core of both implementation programmes, over 50 projects for quality improvement were implemented. These were bottom-up initiated by physicians. Hospital management facilitated project support, including hiring external consultants. Project proposals required an analysis of impact on quality and costs savings. These projects were then evaluated in pilots. A wide range of improvement projects has been implemented. Some involved shared decision-making and decision aids. Others aimed to coordinate and substitute care, such as optometry tests in primary care, or to decrease length of stay in the hospital for specific patient groups.

Mechanism

The bottom-up approach of quality-improvement projects increased the engagement and commitment of physicians that developed their own care projects. Conversely, nurses experienced a lack of engagement. They perceived a stronger focus by the board on stimulating physicians to develop improvement projects. Another barrier to achieving engagement and commitment to quality projects was the administrative burden. All projects had to be presented with a business case demonstrating potential quality gains and savings, which were deemed time-consuming. Nevertheless, this approach was helpful in translating the abstract goal of improving quality to more tangible actions. The envisioned new culture — moving away from production-driven care to quality-driven service — became embedded into the practice of everyday care delivery.

“The projects have contributed to cultural change, in my opinion, because everyone was involved and was having a positive experience. This is motivating and has been an important factor in building a new culture” (i17, hospital manager).

Outcome

A substantial proportion of the care improvement projects in Bernhoven hospital was targeted at shared decision-making. Interviewed patients did not measurably perceive an increase in shared decision-making, but overall patient satisfaction did increase (Supplementary file 3). The Beatrix hospital focused several projects on substituting care towards GPs.

Indicators on quality of care — structure, process and outcome — show predominantly positive trendlines (Figure 1). These indicators improved slightly over the implementation runtime compared to other hospitals.

Close

Figure 1. Trendline Z-Composite Score for (a ) Structure, (b) Process, and (c) Outcome Indicators. Error Bars Indicate the Standard Error.

DID models (Tables 3 and 4) show that means for outcome quality were lower in the intervention group in the pre-intervention period, whereas structure and process quality means were similar to other general hospitals. After intervention, process quality significantly improved (Table 3). Similarly, trendlines for process quality had a significantly steeper slope in the intervention group. There were no clear effects on structural and outcome quality.

Increased Coordination Between Hospital and Primary Care Leads to Substitution of Care

Context

Multiple projects involved substitution of hospital care to primary care providers. This may reduce unnecessary hospital visits and lead to cost reduction.³⁸ Thus, both hospitals closely collaborated with the overarching cooperatives of primary care physicians (care groups). GPs were closely and actively involved in the implementation phase.

Mechanism

An important prerequisite for reducing unnecessary care in the hospital and delivering care closer to home is increased coordination between hospital and GPs. Both hospitals traditionally have strong relations with their respective primary care organisations. This was considered a facilitator for improved coordination and collaboration. GPs stated that their involvement in the development of the care improvement strategies and the implementation programmes increased engagement and trust. Whereas interaction between specialists and GPs was traditionally experienced as hierarchical, the programmes contributed to the perception of communication at peer-level. Through better collaboration and coordination efficiency in the care chain improved, benefiting patients as well.

“Sometimes a brief consultation with the hospital is necessary, but they do not need to repeat the entire patient process …. Our collaboration has improved a lot, to avoid repeating actions. Unnecessary actions put a strain on patients” (i40, GP).

However, as a result of a shifting volume towards primary care, GPs experienced an increased workload. In some cases, GPs were compensated, but in others they also indicated a lack of compensation for their efforts. As GPs play a pivotal role in the strategies, some interviewees stated that they should also be included in the multiyear contracts.

Outcome

The number of unique patients that received care at their GP practice increased, 0.4% in the Beatrix hospital region and 3.5% in the Bernhoven region was reported.³⁹ Combined with reductions in hospital patients, an increase in unique patients may be an indication of substitution. The idea was that substitution of care should not affect quality of care in a negative way. We have collected some evidence on quality of care for specific projects. Patients with diabetes mellitus type 2 and hypothyroidism who were redirected to their GP did show outcomes that were on par with hospital care, although patient satisfaction declined slightly.⁴⁰

Insufficient Use of Data and Support Hinder Quality Improvement

Data and support were identified as barriers to the improvement process. The exemplar was the lack of data infrastructure for coordination and collaboration between hospitals and primary care. Ideally, GPs and medical specialists work in the same electronic health record, but current systems are not compatible, increasing administrative burdens.

The importance of project support was emphasised in both hospitals. At Bernhoven, specific resources were allocated to project support, but these were scaled down during the runtime of the implementation programme. In contrast, in the Beatrix hospital, project support was incorporated at the heart of the strategy, enabling new projects towards the end of the implementation programme.

Financial Security for Physicians Facilitates Shift Towards Quality-Driven Care

Context

In the Netherlands, specialists are either self-employed within a group practice (39%), salaried (49%) or a combination of both (12%).⁴¹ Bernhoven and Beatrix hospital mainly rely on self-employed physicians. Both hospitals altered physician payment structures. Hospital management and medical staff in Bernhoven decided for the boldest approach: self-employed doctors became salaried physicians, aiming to eliminate most production incentives.^16,42 Physicians in the Beatrix hospital remained self-employed, but their income was no longer tied to volume; physicians were guaranteed a base income and were rewarded for engaging in quality-improvement initiatives (for up to 10% of their income). In both hospitals, physicians would not be negatively financially affected when volumes decreased.

Mechanism and Outcome

Financial incentives for physicians were altered in both hospitals. Several respondents indicated that detaching income from production can incentivise for a reduction in low-value care. Some stated that now they were no longer affected in their professional assessment by financial incentives.

“It was easier to not admit patients into the hospital when doctors were no longer dependent on production for their income …. In contrast, doctors used to compete each other for a patient’s admission because their department would be remunerated” (i4, MD and hospital manager).

Physicians experienced financial security, which accelerated improvement projects aimed at reducing low-value care. Nevertheless, the transition from self-employed to salaried status initially met with high resistance at Bernhoven. Physicians feared loss of power and an increased dependency on hospital management. The self-developed concept of medical leadership, where doctors combined clinical and managerial functions, was thought to prevent such dependence. Physicians at Beatrix hospital indicated that they now were better incentivised to improve quality since the new redistribution model rewarded them for doing so.

Scaling Down Hospital Facilities Is Required

Context

Bernhoven incorporated a new organisational model.² Instead of structures emphasising the different medical specialties, four business units were created based on workflow processes: (1) diagnosis and decision making, (2) intervention unit, (3) acute care unit, and (4) a chronic care unit.¹⁶ The assumption was that organising the hospital according to workflow functions would harmonize processes of care and, ultimately, lead to efficient service delivery. For example, blocks of planned surgery appointments could be scheduled more efficiently, and acute care was strengthened with additional medical expertise to avoid unnecessary downstream costs. Beatrix did not restructure their organisational model, but aimed to scale down internal cost structures.

Mechanism

To reduce costs, the decreasing volumes of care eventually require reducing the cost-structure of the hospital. However, scaling down (semi) fixed-costs takes time and thus the savings lag actual volume reductions. Combining quality initiatives with reorganising workflow processes might allow scaling down capacity. For example, Bernhoven’s reorganisation into four business units was perceived to contribute to a more streamlined care delivery process.

“The business units fit the philosophy and culture of the Dream programme: a shared responsibility among departments benefiting the patient with a more holistic view” (i17, hospital manager).

Although the Beatrix hospital did not fundamentally reorganise its core processes, substantial efforts were made to reduce fixed costs. An entire nursing ward was closed as a result of fewer hospitalisations. However, significant drops in volume are necessary to be able to close an entire ward and the limited scale of these regional hospitals is a barrier since a minimum necessary scale exists.

“A barrier for small hospitals is the amount of fixed costs that cannot be reduced…. Some departments cannot be scaled down in terms of staffing if you wish to have a continuous service available. Surgeons and their subspecialties, need sufficient staffing” (i12, MD and hospital manager).

Outcome

Annual reports for Bernhoven show reductions in operating costs relative to other hospitals (Figure 2). While capital costs have declined, personnel expenditure demonstrate a relative increase in 2020, compared to other hospitals. Variable care-related costs witness substantial decline from a top in 2017 which correlates with the lower volumes. As Beatrix hospital is part of a larger organisation that also includes long-term-care provision (Rivas), these trendlines cannot be isolated for this hospital.

Close

Figure 2. Bernhoven Operating Expenditure Growth Relative to the Median of All Hospitals (2012 Base Year). Total costs are a sum of personnel, capital, general, and care-related costs.

Multiyear Shared Savings Arrangements Under Global Budgets May Increase Alignment Between Payers and Hospitals

Context

In the Netherlands, hospital payment is case-based (diagnosis−treatment combinations).⁴¹ Price and volumes of these diagnosis−treatment combinations are negotiated between insurance companies and providers.⁴³ It is assumed that this may provide doctors with incentives for overutilization, leading to unnecessary care volumes. Bernhoven and Beatrix hospital came up with alternatives. They formed an alliance with the largest healthcare insurers in their areas and complied to a multiyear global budget contract with a runtime of five years.¹⁶ This contract included accountability clauses to ensure accessibility and quality of care (ie, to prevent undertreatment). The contract was based on a shared savings rationale (Figure 3): insurers pay for necessary upfront investments followed by multiyear flat real growth. Over the contract period, payers pay less than trendline cost growth and thus they collect savings. The multiyear agreement provides the hospitals with financial security while transforming their organisation to reduce their structural cost base. Hospitals may benefit financially if they succeed in reducing (fixed) hospital costs and thus collect their part of the shared savings rationale.

Close

Figure 3. Conceptual Model Multiyear Global Budget.

Mechanism

The multiyear global budget provided financial security for structural changes in both hospitals. Both strategies required upfront financial investments, which were incorporated into the arrangement. Thinking was that as the strategies aimed to reduce volumes, revenue streams would over time start to decline. Financial security should also engage physicians. The psychological basis of the arrangement between health insurers and hospitals was built on a win-win perspective in which savings were ‘equally’ shared. Cooperative mentality was perceived as a step forward from preceding rivalrous negotiations.

“The programme truly gained momentum when two main insurers were committed to the agreement: cost savings would be shared and reallocated to benefit policy holders and reinvested into the hospital …. This created a sense of partnership as opposed to the insurer cream-skimming the hospitals’ efforts” (i36, healthcare insurer).

However, the room for volume and cost reductions is not infinite. Besides, uncertainty over upcoming negotiations was present among several stakeholders toward the end of the contract period. This increased tensions between health insurers and the hospital (in the case of Bernhoven). Power asymmetry exists (Ratchet effect): if the insurer decides to return to case-based reimbursements, the hospital might practically not be able to return to former levels of production, threatening the financial viability of the organisation.

Outcome

An important aim was to reduce low-value care. Both Bernhoven and Beatrix hospital managed to decrease their volumes of care delivery compared to a control group of hospitals. The Beatrix hospital decreased its relative volume by 7% in 2017; Bernhoven achieved a whopping 13% decrease.³⁹ Additional analyses showed no indication of patients in either region visiting other hospitals (spillover effects).³⁹ A key factor is the extent to which such substantial volume reductions result in reductions in operating costs.

“We managed to reduce the volume of care, but did we also save costs? The health insurers are saying, ‘The hospital is still too expensive, since the agreed cost reductions were not reached’” (i35, MD and hospital manager).

Multiyear global budgets did not lead to reductions in hospital revenue (ie, insurer costs) in the average of both intervention hospitals, compared to other Dutch hospitals (Figure 4). Hospital revenue showed a relative increase in the early years of implementation (upfront investments) after which costs returned to, but not underneath, the level of the other hospitals. This indicates that from the payer perspective, the strategy did not significantly result in lower relative hospital expenses, despite reductions in volume of care. However, the latter might actually lead to a reduction in their own payments from the risk-adjustment funds that partly relate to hospital volume. The profit margins at Bernhoven also were on a negative path, but Rivas (Beatrix hospital) shows an upward trend since 2016 (Supplementary file 4). The latter does includes the whole care group and we cannot separate hospital results from those of the elderly care units.

Close

Figure 4. Hospital Revenue (ie, Insurer Expenses) of Both Intervention Hospitals Compared to Counterfactual Based All Other Hospitals in the Netherlands.

Tables 3 and 4 show DID models estimating the effects on hospital revenue (insurer-based expenditures), solvency ratio, and profit margins. Insurer-based expenditures, total expenditures and solvency ratios were lower in the intervention group as compared to other general hospitals (pre-intervention). No significant differences pre-intervention indicate that the parallel trends assumption holds.³³ The intervention group experienced lower mean expenditures in the post intervention period, but not significantly. Similar results ensue from the DID analyses estimating trends: Baseline values were lower in the intervention group, and trends follow a reduced expenditure growth rate, but not significantly (Table 4). Solvency ratios and profit margins demonstrated no clear differences in trends. These results are qualitatively similar to the trend analyses.

Discussion

This study hypothesized six pivotal mechanisms that potentially contribute to quality improvement and cost reduction. First, actual improvements in patient care were achieved through active, bottom-up engagement of professionals in multiple quality projects. Physicians developed and initiated quality-improvement projects that contribute to a strong base of support. Second, both hospitals also managed to substitute care to GPs; communication between primary and secondary care was considered positive and improved. Third, insufficient use of data and support hinders quality improvement. This barrier was apparent in the lack of data infrastructure for primary and hospital care coordination as well as resources for data support. Fourth, in both hospitals, reducing financial incentives for physicians to increase the volume of care was deemed important. Stakeholders perceived a greater focus on quality of care, opposed to volumes. Fifth, scaling down hospitals may convert volume reductions to costs savings. Although both hospitals managed to scale down their organisation, additional efforts are needed to keep pace with volume reductions. This seems to be especially the case at Bernhoven, which witnessed declining profit margins. Sixth, multiyear global budget agreements between payers and hospitals may contribute to decelerating volume growth through financial security. However, concerns are raised whether this strategy is sustainable beyond the contract period, as a result of a lag in the reduction of internal costs.

Culture was shifted towards quality-based and patient-centred care through a combination of elements of the strategy, such as quality-improvement projects, financial incentives, and culture workshops. The institution-wide implementation of both strategies contributed to their success. Hospital performance was improved through engagement of all stakeholders, facilitating the evolution of a multiyear implementation programme, and strong clinical and managerial leadership.

Strengths and Limitations

The main strength of this study is triangulation of data sources. This allowed us to evaluate hospital strategies that transcend traditional siloes, such as primary and secondary care and provider-payer divisions. The integrated use of methods allowed us to draw conclusions beyond simple outcomes such as cost and quality of care.⁷ However, this does not imply that our findings, as described in six mechanisms, can be used as blueprints for all future hospital improvement programmes. These mechanisms are useful but, ultimately success is related to the engagement of stakeholders within the system and strategy fit to the local context.

An important limitation to our study is that net effects are difficult to discern. As these strategies are complex and involve multiple elements at different levels within and outside the organisation, we cannot assume causality between specific outcomes we observe and specific elements. Furthermore, our data on costs and quality outcomes had some limitations.

Explanatory power of our DID models was relatively low, due to the limited number of observations (approximately 70 hospital organisations) in the Netherlands. Furthermore, the start of the improvement programme defined the pre- and post-intervention periods, whereas these interventions were multifaceted (eg, not all quality improvement projects were initiated during the first year). Nevertheless, the explanatory analyses were in line with descriptive trends, ie, insignificantly lower expenditure growth, no evident effects on other financial indicators and limited, positive effects on quality between the intervention hospitals and other hospitals were observed. Given the variance within the control group, the Dutch hospital sector may have been inherently underpowered to detect real effects on quality or costs. Moreover, patient satisfaction scores were subject to selection bias. Some elements are difficult to capture in indicators such as patient-centredness. Furthermore, available quality indicators were subject to large variation between years, which limited comparisons over time for individual variables and specific sets of indicators. To correct this, we constructed an average benchmark z-score. This benchmark score may level out positive and negative outliers. In theory, variance in quality indicators may increase as a result of the strategic programs, but this was beyond the scope of this research. Therefore, effects on quality should be interpreted with caution. Although we interviewed patients, it was difficult to ascribe personal experiences to specific mechanisms. Nevertheless, the broad spectrum of data sources and indicators we used to an extent do mitigate such limitations. The interpretation of results should be integrated, rather than based on individual data streams and analyses.

Previous Research

While several studies have evaluated improvement programmes that target single departments or patient groups, evaluations of institution-wide strategy implementations are much more scarce.^7,8,10,44-46 Nevertheless, earlier research on organization-wide hospital strategies demonstrated that engagement of staff at all levels was found to be critical for successful implementation.^8,47-50 These strategies often translate into a multitude of projects on the operational level of hospitals, which can further enhance staff engagement.^9,46

Furthermore, a recent systematic review found substitution of hospital care to specialist services in primary care settings to have a positive effect on quality of care.⁵¹ Although only some projects in Bernhoven and the Beatrix hospital involved this type of substitution, results were similar in terms of costs and quality. The qualitative part of our study also adds that increased coordination between hospital and primary care can facilitate efficient care delivery in both settings.^51,52

Evidence on appropriate payment models for specialist physicians remains limited.⁴² The effectiveness of physician payment models seems to depend more on adequate quality and performance indicators than the payment model itself (salaried versus self-employed).⁴² Physician payment models were not the main focus of our study, but both hospitals used different payment models resulting in similar experiences: stakeholders perceived an increased focus on quality rather than volume of care. Another case study on institution-wide hospital improvement showed that cost savings are not generated instantly, but take some time to take effect.⁷ Furthermore, alignment of payers and providers through alternative payment models may be beneficial for increasing quality and reducing costs.⁵³ Moreover, global budgets were relatively successful in reducing healthcare utilisation in the Netherlands.⁵⁴ Existing case studies underline the importance of accurate and timely data to support change processes,^7,44 which seemed insufficiently acknowledged in the strategies of Bernhoven and the Beatrix hospital.

The context of both hospitals was described extensively in this study. Previous studies on hospitals in the Netherlands have illustrated the importance of a local approach rather than using a blueprint strategy.^45,46 Bernhoven and Beatrix showed clear contextual similarities and differences, but both were able to adjust a similar improvement rationale to local needs.

Traditional concepts of quality improvement are considered linear interventions with a causal effect on outcomes, indicating that repeated interventions would lead to similar results. The results in this study demonstrate the importance of contextual factors. Improvement is not a single intervention in time but “an interdependent set of actions.”¹² This study adds real-world evidence to the limited body of knowledge on hospital improvement strategies.

Implications

Hospital-wide approaches for quality improvement and limiting costs may be successful. This study demonstrated that a convergent parallel mixed methods approach is useful for evaluating complex hospital interventions. Although context remains important for success, several general mechanisms were identified. We discern several implications for both policy-makers (macro-level) and hospital managers (meso-level).

For hospital managers, a broad scope in engaging professionals, such as doctors and nurses, may improve effectiveness of the strategy on the operational level of the organisation. Hospital leadership should facilitate iterative cycles of improvement based on mutual goals. A drastic reorganisation of the hospital structure may be required to convert volume reductions into cost savings. The theoretical mechanism of shared savings appears, in practice, to require a delicate balance between a relative decrease in revenue and scaling down internal hospital costs.

External policy reform may accelerate the shift towards appropriate care through alternative payment models, moving away from production-based incentives. Moreover, primary care providers may require compensation for additional burdens of substitution. Alternative or population-based payment models could provide fair allocation of resources, if risks for undertreatment and inefficiency can be restricted. This may aid the process of moving away from traditional siloes in healthcare, towards integrated care delivery.

Recently, Bernhoven faced financial distress. The hospital was at the centre of the first wave of COVID-19 infections. In comparison to larger hospitals, smaller hospitals got fewer resources to compensate for additional costs. However, our analysis shows that the internal cost-structure of this hospital was already behind on its financial goals before the pandemic. The challenge of reorganising and scaling down hospital facilities should have a prominent emphasis in future strategies that aim to contribute to appropriate care. Our analyses showed that internal costs reductions lag volume reductions, resulting in relatively costly treatments. This may incentivise insurers to cut treatment prices and capitalize on lower levels of production.⁵⁵ This ratchet effect is a concern for the longer-term sustainability of these strategies.

We recommend an integrated approach to future research on hospital improvements. Baseline assessments of quality indicators, financial situation, and stakeholders’ perceptions before, during and after the runtime of the strategy implementation programme strengthen analyses. Longer-term effects of such programmes are currently unknown and research could offer important insights into longer-term sustainability. Further research may build on our findings to analyse how to consolidate and adapt improvement processes and mechanisms in the longer term.

Conclusion

We conducted an integrated analysis of mixed data sources to evaluate complex hospital improvement programmes. Two hospitals managed to (temporally) break the trend of annually increasing volumes, while maintaining quality of service delivery. However, long-term financial outcomes of these strategies raise concerns, since short-term cost reductions for both hospitals (substantially) lag behind volume reductions. Additional reorganisation efforts within and outside the hospital to scale down internal costs seem necessary to perpetuate the pursuit of improvement. These strategies underline the necessity of an approach customised to local context and which is subject to iterative improvement.

Ethical issues

Ethical approval was obtained at the Medical Research Ethics Committee Arnhem-Nijmegen, the Netherlands (Ref. 2019-5353).

Competing interests

JK was part of the transformation of both hospitals as external strategic advisor; no other relationships or activities that could appear to have influenced the submitted work.

Funding

This work was supported by the Ministry of Health, Welfare and Sport, The Hague, The Netherlands.

Supplementary files

References

1. Jeurissen P, Duran A, Saltman RB. Uncomfortable realities: the challenge of creating real change in Europe’s consolidating hospital sector. BMC Health Serv Res 2016; 16(Suppl 2):168. doi: 10.1186/s12913-016-1389-3 [Crossref] [ Google Scholar]
2. Christensen CM, Grossman JH, Hwang J. The Innovator’s Prescription. New York: McGraw-Hill; 2010.
3. Berwick DM, Hackbarth AD. Eliminating waste in US health care. JAMA 2012; 307(14):1513-1516. doi: 10.1001/jama.2012.362 [Crossref] [ Google Scholar]
4. Marshall M, Øvretveit J. Can we save money by improving quality?. BMJ Qual Saf 2011; 20(4):293-296. doi: 10.1136/bmjqs.2010.050237 [Crossref] [ Google Scholar]
5. Øvretveit J. Does Improving Quality Save Money? A Review of Evidence of Which Improvements to Quality Reduce Costs to Health Service Providers. London: The Health Foundation; 2009. p. 95.
6. Schouten LM, Hulscher ME, van Everdingen JJ, Huijsman R, Grol RP. Evidence for the impact of quality improvement collaboratives: systematic review. BMJ 2008; 336(7659):1491-1494. doi: 10.1136/bmj.39570.749884.BE [Crossref] [ Google Scholar]
7. Chatfield SC, Volpicelli FM, Adler NM. Bending the cost curve: time series analysis of a value transformation programme at an academic medical centre. BMJ Qual Saf 2019; 28(6):449-458. doi: 10.1136/bmjqs-2018-009068 [Crossref] [ Google Scholar]
8. Nilsson K, Bååthe F, Erichsen Andersson A, Sandoff M. Value-based healthcare as a trigger for improvement initiatives. Leadersh Health Serv (BradfEngl) 2017; 30(4):364-377. doi: 10.1108/lhs-09-2016-0045 [Crossref] [ Google Scholar]
9. Jabbal J, Lewis M. Approaches to Better Value in the NHS: Improving Quality and Cost. King’s Fund; 2018.
10. Wackers E, Stadhouders N, Heil A, Westert G, van Dulmen S, Jeurissen P. Hospitals bending the cost curve with increased quality: a scoping review into integrated hospital strategies. Int J Health Policy Manag 2021; 11(11):2381-2391. doi: 10.34172/ijhpm.2021.168 [Crossref] [ Google Scholar]
11. Li SA, Jeffs L, Barwick M, Stevens B. Organizational contextual features that influence the implementation of evidence-based practices across healthcare settings: a systematic integrative review. Syst Rev 2018; 7(1):72. doi: 10.1186/s13643-018-0734-5 [Crossref] [ Google Scholar]
12. Øvretveit J. Understanding the conditions for improvement: research to discover which context influences affect improvement success. BMJ Qual Saf 2011; 20(Suppl_1):i18-23. doi: 10.1136/bmjqs.2010.045955 [Crossref] [ Google Scholar]
13. Greenhalgh T, Humphrey C, Hughes J, Macfarlane F, Butler C, Pawson R. How do you modernize a health service? A realist evaluation of whole-scale transformation in London. Milbank Q 2009; 87(2):391-416. doi: 10.1111/j.1468-0009.2009.00562.x [Crossref] [ Google Scholar]
14. Nurjono M, Shrestha P, Lee A. Realist evaluation of a complex integrated care programme: protocol for a mixed methods study. BMJ Open 2018; 8(3):e017111. doi: 10.1136/bmjopen-2017-017111 [Crossref] [ Google Scholar]
15. Moore GF, Audrey S, Barker M. Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015; 350:h1258. doi: 10.1136/bmj.h1258 [Crossref] [ Google Scholar]
16. van Leersum N, Bennemeer P, Otten M, Visser S, Klink A, Kremer JAM. Cure for increasing health care costs: the Bernhoven case as driver of new standards of appropriate care. Health Policy 2019; 123(3):306-311. doi: 10.1016/j.healthpol.2019.01.002 [Crossref] [ Google Scholar]
17. Visser S, Westendorp R, Cools K, Kremer J, Klink A. KwaliteitalsmedicijnAanpakvoorbeterezorgenlagerekosten. Booz & Company; 2012.
18. Venhuizen G. Can patient centred care plus shared decision making equal lower costs?. BMJ 2019; 367:l5900. doi: 10.1136/bmj.l5900 [Crossref] [ Google Scholar]
19. Polak L. Bernhoven approach is more than just patient centred care and shared decision making. BMJ 2019; 367:l6560. doi: 10.1136/bmj.l6560 [Crossref] [ Google Scholar]
20. Campbell M, Fitzpatrick R, Haines A. Framework for design and evaluation of complex interventions to improve health. BMJ 2000; 321(7262):694-696. doi: 10.1136/bmj.321.7262.694 [Crossref] [ Google Scholar]
21. Skivington K, Matthews L, Simpson SA. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ 2021; 374:n2061. doi: 10.1136/bmj.n2061 [Crossref] [ Google Scholar]
22. Curry LA, Krumholz HM, O’Cathain A, Plano Clark VL, Cherlin E, Bradley EH. Mixed methods in biomedical and health services research. Circ Cardiovasc Qual Outcomes 2013; 6(1):119-123. doi: 10.1161/circoutcomes.112.967885 [Crossref] [ Google Scholar]
23. Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implement Sci 2013; 8:51. doi: 10.1186/1748-5908-8-51 [Crossref] [ Google Scholar]
24. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009; 4:50. doi: 10.1186/1748-5908-4-50 [Crossref] [ Google Scholar]
25. Croes RR, Krabbe-Alkemade Y, Mikkers MC. Competition and quality indicators in the health care sector: empirical evidence from the Dutch hospital sector. Eur J Health Econ 2018; 19(1):5-19. doi: 10.1007/s10198-016-0862-6 [Crossref] [ Google Scholar]
26. Zorginstituut Nederland. Transparantiekalender. https://www.zorginzicht.nl/transparantiekalender. Accessed November 2020.
27. Donabedian A. The quality of care. How can it be assessed? JAMA 1988; 260(12):1743-1748. doi: 10.1001/jama.260.12.1743 [Crossref] [ Google Scholar]
28. ZorgkaartNederland. https://www.zorgkaartnederland.nl/. Accessed November 2021.
29. CIBG. Jaarverantwoording in de Zorg. https://digimv8.desan.nl/archive/search.
30. Diepstraten M, Douven R, Kopányi-Peuker A, Wijnker G. CasestudieBeatrixziekenhuisenBernhoven. Den Haag: CPB; 2020.
31. Tsai TC, Orav EJ, Jha AK. Patient satisfaction and quality of surgical care in US hospitals. Ann Surg 2015; 261(1):2-8. doi: 10.1097/sla.0000000000000765 [Crossref] [ Google Scholar]
32. Austin JM, D’Andrea G, Birkmeyer JD. Safety in numbers: the development of Leapfrog’s composite patient safety score for US hospitals. J Patient Saf 2014; 10(1):64-71. doi: 10.1097/PTS.0b013e3182952644 [Crossref] [ Google Scholar]
33. Dimick JB, Ryan AM. Methods for evaluating changes in health care policy: the difference-in-differences approach. JAMA 2014; 312(22):2401-2402. doi: 10.1001/jama.2014.16153 [Crossref] [ Google Scholar]
34. Ryan AM, Burgess JF Jr, Pesko MF, Borden WB, Dimick JB. The early effects of Medicare’s mandatory hospital pay-for-performance program. Health Serv Res 2015; 50(1):81-97. doi: 10.1111/1475-6773.12206 [Crossref] [ Google Scholar]
35. Mishra A, DeLia D, Zeymo A. ACA Medicaid expansion reduced disparities in use of high-volume hospitals for pancreatic surgery. Surgery 2021; 170(6):1785-1793. doi: 10.1016/j.surg.2021.05.033 [Crossref] [ Google Scholar]
36. Farrar S, Yi D, Sutton M, Chalkley M, Sussex J, Scott A. Has payment by results affected the way that English hospitals provide care? Difference-in-differences analysis. BMJ 2009; 339:b3047. doi: 10.1136/bmj.b3047 [Crossref] [ Google Scholar]
37. Krabbe-Alkemade YJ, Groot TL, Lindeboom M. Competition in the Dutch hospital sector: an analysis of health care volume and cost. Eur J Health Econ 2017; 18(2):139-153. doi: 10.1007/s10198-016-0762-9 [Crossref] [ Google Scholar]
38. van Hoof SJM, Quanjel TCC, Kroese M, Spreeuwenberg MD, Ruwaard D. Substitution of outpatient hospital care with specialist care in the primary care setting: a systematic review on quality of care, health and costs. PLoS One 2019; 14(8):e0219957. doi: 10.1371/journal.pone.0219957 [Crossref] [ Google Scholar]
39. Netherlands Bureau for Economic Policy Analysis (CPB), Dutch Healthcare Authority (NZa), IQ healthcare. Evaluation Programmes Beatrix Hospital and Bernhoven. CPB; 2020.
40. Van Dulmen S, Stadhouders N, Westert G, Wackers E, Jeurissen P. Op wegnaarhogekwaliteitenlagekosten in de medischspecialistischezorg. Evaluatieveranderprogramma’s Rivas ZorggroepenBernhoven. Nijmegen: IQ healthcare; 2020.
41. Wammes J, Stadhouders N, Westert G. International Health Care System Profiles: Netherlands. https://www.commonwealthfund.org/international-health-policy-center/countries/netherlands. Accessed March 12, 2021. Published June 5, 2020.
42. Quinn AE, Trachtenberg AJ, McBrien KA. Impact of payment model on the behaviour of specialist physicians: a systematic review. Health Policy 2020; 124(4):345-358. doi: 10.1016/j.healthpol.2020.02.007 [Crossref] [ Google Scholar]
43. Douven R, Burger M, Schut F. Does managed competition constrain hospitals’ contract prices? Evidence from the Netherlands. Health Econ Policy Law 2020; 15(3):341-354. doi: 10.1017/s1744133119000215 [Crossref] [ Google Scholar]
44. Lee VS, Kawamoto K, Hess R. Implementation of a value-driven outcomes program to identify high variability in clinical costs and outcomes and association with reduced cost and improved quality. JAMA 2016; 316(10):1061-1072. doi: 10.1001/jama.2016.12226 [Crossref] [ Google Scholar]
45. van den Heuvel J, Does RJ, Bogers AJ, Berg M. Implementing six sigma in the Netherlands. Jt Comm J Qual Patient Saf 2006; 32(7):393-399. doi: 10.1016/s1553-7250(06)32051-x [Crossref] [ Google Scholar]
46. Niemeijer GC, Trip A, de Jong LJ, Wendt KW, Does RJ. Impact of 5 years of lean six sigma in a University Medical Center. Qual Manag Health Care 2012; 21(4):262-268. doi: 10.1097/QMH.0b013e31826e74b7 [Crossref] [ Google Scholar]
47. Barnabè F, Guercini J, Perna MD. Assessing performance and value-creation capabilities in Lean healthcare: insights from a case study. Public Money Manag 2019; 39(7):503-511. doi: 10.1080/09540962.2019.1598197 [Crossref] [ Google Scholar]
48. Strongwater SL. The change process and a clinical evaluation unit at University of Massachusetts Medical Center. Qual Manag Health Care 1996; 4(4):30-39. doi: 10.1097/00019514-199604040-00005 [Crossref] [ Google Scholar]
49. Rees C. How Hospitals Can Save 18 Percent in Direct Costs. Children’s Hospitals Today. July 13, 2014.
50. Birk S. Quality, cost efficiency, the new quality-cost imperative: systemwide improvements can yield financial gains. World Hosp Health Serv 2010; 46(2):16-19. [ Google Scholar]
51. van Hoof SJM, Quanjel TCC, Kroese M, Spreeuwenberg MD, Ruwaard D. Substitution of outpatient hospital care with specialist care in the primary care setting: a systematic review on quality of care, health and costs. PLoS One 2019; 14(8):e0219957. doi: 10.1371/journal.pone.0219957 [Crossref] [ Google Scholar]
52. van Hoof SJ, Kroese ME, Spreeuwenberg MD. Substitution of hospital care with primary care: defining the conditions of Primary Care Plus. Int J Integr Care 2016; 16(1):12. doi: 10.5334/ijic.2446 [Crossref] [ Google Scholar]
53. Remers TEP, Wackers EME, van Dulmen SA, Jeurissen PPT. Towards population-based payment models in a multiple-payer system: the case of the Netherlands. Health Policy 2022; 126(11):1151-1156. doi: 10.1016/j.healthpol.2022.09.008 [Crossref] [ Google Scholar]
54. Gaspar K, Portrait F, van der Hijden E, Koolman X. Global budget versus cost ceiling: a natural experiment in hospital payment reform in the Netherlands. Eur J Health Econ 2020; 21(1):105-114. doi: 10.1007/s10198-019-01114-6 [Crossref] [ Google Scholar]
55. Brekke KR, Siciliani L, Straume OR. Hospital competition with soft budgets. Scand J Econ 2015; 117(3):1019-1048. doi: 10.1111/sjoe.12111 [Crossref] [ Google Scholar]