Portuguese Primary Healthcare and Prevention Quality Indicators for Diabetes Mellitus – A Data Envelopment Analysis

Background

Over the last decades, we have seen a great effort by governments and health authorities to improve overall quality of health services, especially in primary healthcare (PHC), the pillar of the entire health system.^1-3 This is a relevant concern for policy-makers as better primary care services, which comprises health promotion, prevention and monitoring of health-disease processes and the natural history of co-morbidities, have an impact on overall populations’ health, whether they are favourable or not.^4-6 A study was carried out previously by the authors that synthesised and categorised the quality indicators frequently used and monitored by PHC, published in peer-reviewed studies indexed in bibliographic databases.⁷ Quality indicators are a set of measures that are especially useful in monitoring processes and results, controlling compliance with best clinical practices through quantitative parameters.^8,9 Thus, this monitoring through indicators aims at better health processes and outcomes. The growing number of patients suffering from chronic diseases,^10,11 mostly as a result of aging populations, have forced healthcare systems to address increasing costs and demand for care. An important share of healthcare spending is considered ineffective and wasteful, mostly due to ambulatory care sensitive conditions or avoidable hospitalizations. A large number of hospital admissions could be avoided through better prevention and management of chronic conditions at primary care level. Amongst more than 30 conditions for which hospitalization could be reduced with improved primary care,¹² diabetes mellitus (DM) stand out as particularly relevant in European countries, being responsible for more than 800 000 admissions in the European Union in 2015, consuming more than 6.7 million bed days (1.1% of all bed days).¹³

DM is a chronic disease where the need of higher quality of primary care is paradigmatic,¹⁴ due to the worldwide high prevalence, the inherent complexity that implies greater proximity to the patient, and the possibility to prevent most of the complications causing hospitalization.^15-18

In Portugal, 9.2% of the population lived with DM in 2016, which caused 5% of all deaths.^16,19,20 Therefore, tracking preventable hospitalizations due to diabetes is a key strategy to indirectly evaluate primary care performance regarding this disease. The Agency for Healthcare Research and Quality developed the prevention quality indicators (PQIs), which consist in a set of measures that can be used with hospital discharge data to identify the performance of primary care regarding the treatment and management of ambulatory care sensitive conditions.^7,21-25 Despite recent improvements, preventable hospitalizations rates due to diabetes in Portugal remain high and vary substantially across local and regional providers, hence monitoring is of utmost importance. The total number of hospital admissions due to diabetes complications was 8.5 admissions in 2016, decreasing to 6.9 admissions in 2017, with long term diabetes complications being the main contributor to these results (4012 hospitalizations in 2016 and 2639 hospitalizations in 2017).²⁶ However, during that period, hospitalizations due to uncontrolled diabetes and lower-extremity amputation among patients with diabetes complications increased, indicating that some areas of diabetes care need to be improved. As the performance of PHC reflects onto hospitalizations for conditions sensitive to this level of care,^27-29 thus efficiency in primary care can reduce this type of hospital admissions, thereby reducing wasteful spending on healthcare.

In 1999 the Institute of Medicine defined quality of care as “the degree to which health services increase the likelihood of desired results and are consistent with current professional knowledge,”³⁰ and categorised the domains of health quality in 2001 into six pillars: safe, effective, efficient, timely, patient-centred, and equitable.^31,32 We note, however, that efficiency assessment specially in PHC is still an under-explored pillar.

Over the last decades, several techniques to perform efficiency analysis in healthcare have been proposed,³³ with a predominant use of data envelopment analysis (DEA) in nearly all healthcare sectors.³⁴ Regarding primary care, DEA has increasingly been employed, showing its suitability for this setting.³⁵ DEA is a non-parametric technique that was first presented in the study “Measuring the efficiency of decision-making units (DMU)” in 1978.³⁶ Several studies were subsequently carried out, mainly in the areas of management, engineering and healthcare, the last one often with few consensuses regarding the most appropriate method.^37-44 Nevertheless, authors prefer the use of DEA methods due to many underlying advantages, namely the possibility of considering multiple inputs and outputs, which better characterize the health production process, the fact that it does not require a mathematical specification or assumptions regarding the production function, apart from being the most appropriate approach to assess the impact of exogenous variables and for providing means to draw recommendation and potential of improvement for inefficient units.⁴⁵ Healthcare efficiency has been increasingly assessed over time and more recently has been considered a relevant quality indicator for healthcare assessment, including PHC.^7,46 Some studies were carried out involving the analysis of efficiency through DEA, specifically in the context of DM, but did not evaluate the unfavourable outcome (preventable hospitalizations) as an output.^43,47-51 Since efficiency can be considered a polysemantic word, in this study we considered the definition of efficiency as “the capacity or even the ability to make the most appropriate use of the resources available to achieve an intended result.”^52,53 To fill this gap, our study aims to evaluate the efficiency of primary care in Portugal regarding diabetes care, considering the broadest perspective of policy-makers and public health managers, that is, how to use the available resources to reduce avoidable hospitalizations due to DM, which represents a proxy of overall PHC quality concerning this particular disease.

Methods

Data Sources and Variables

The Portuguese PHC is backed up by a robust information system, producing a comprehensive set of primary care activity and performance indicators that are made publicly available by the Ministry of Health at the Bilhete de Identidade dos Cuidados de Saúde Primários website, and the definitions of these indicators can be found elsewhere.^20,55 Data on human resources to be used as input variables to the DEA models, as well as variables for Tobit analysis were extracted from these sources. In mainland Portugal, the primary care is organized into five regions called Health Regional Administration, which are responsible for tackling the health needs of the population living in their area of intervention, including resource management and actions to ensure the access and quality of health services. Each health regional administration is composed of several primary care health centre groupings denominated ACES (Agrupamentos de Centros de Saúde), which in turn provide care to the population living within a specific geographical area of intervention, usually at municipality level.

To estimate diabetes-related PQI rates, we used patient level data abstracted from the Portuguese National Hospital Morbidity Database, which contains data on inpatient and outpatient episodes that occurred in all public hospitals within the Portuguese National Health Service. Coded clinical data from all inpatient episodes with a discharge date between January 1, 2016 and December 31, 2017 were extracted. Variables included diagnoses and procedures coded according to the International Classification of Diseases, Clinical Modification, ninth and tenth revisions (ICD-9-CM/ICD-10-CM); age; sex; discharge status and place of residence. The latter variable was included to assign each inpatient episode to an ACES. We identified avoidable hospitalizations according to the definitions of all 5 DM-related PQIs reported in the version 6.0 for ICD-9-CM codes and version 2019 for ICD-10-CM/PCS codes, both available at the US Agency for Healthcare Quality documentation.²³ Age-sex standardized PQI rates were calculated for each ACES using the direct standardization method, considering population by age and sex at parish level collected from the 2011 Census, which is the latest population census available by the Portugal’s National Institute of Statistics.⁵⁶

The variables analyzed in this study include Input variables (number of doctors, number of nurses and number of intern doctors), Output variables (PQI 01: Diabetes Short-Term Complications Admission Rate, PQI 03: Diabetes Long-Term Complications Admission Rate, PQI 14: Uncontrolled Diabetes Admission Rate, PQI 16: Lower-Extremity Amputation Among Patients with Diabetes Rate, and PQI 93: Prevention Quality Diabetes Composite), and the Environmental variables (Ratio Patients by Medical Doctors, Proportion of elderly patients, Unemployment rate, Prevalence of DM, Proportion of patients with DM and record of hemoglobin A1c (HbA1c) in the last six months, Proportion of patients with DM and with last HbA1c ≤8.0%, Proportion of patients with DM <65 years old with a HbA1c test ≤6.5%, Proportion of patients with DM and with a microalbuminuria test in the last year, Proportion of patients with a risk assessment for DM2 (3 years), Cost of therapy for diabetic patients, and Cost of therapy for diabetic patients under control). Before describing the employed chosen model, it is worth mentioning that efficiency can be assessed in health outcomes and health outputs.^31,33 Specifically, our work assesses the efficiency based on health outcomes in primary care, which are usually represented by preventable admissions and which, as already explained, are usually considered as a quality indicator of PHC. Efficiency was analysed, particularly in the context of primary care related to diabetes care.

Data Envelopment Analysis Model

In DEA efficiency assessments, the units of analysis are denominated DMU, which in our study are represented by the 55 ACES in mainland Portugal. The relative efficiency of ACES was evaluated by means of multiple inputs (PHC resources) and multiple products (PHC outcomes). The objective of our DEA model, therefore, is to compare the 55 ACES that perform similar tasks (PHC) and differ in the quantities of inputs (human resources) they provide and the outputs they produce (diabetes PQIs results). Nevertheless, ACES Pinhal Interior Sul was excluded from analysis as it operates under different conditions in comparison with the other ACES, namely due to the smaller size of the population covered, and for being the only ACES that do not present intern doctors in its human resource composition. Therefore, the final sample comprises 54 DMUs.

Relevant points to choose the DEA model, among others, are the possibility of the model to deal with the existence of several non-proportional inputs and outputs; to use all available data in order to build an empirical frontier of best practices, in which each production point classified as “not optimal” is compared; it does not require the specification of the functional form that connects inputs to outputs; and the performance of the units can be assessed through several alternative guidelines for the frontier of best practices, depending on the context of the study. Despite these advantages, the model can also bring some disadvantages. These were considered when conducting this study and mitigated by the close engagement with the practices. Disadvantages include the fact that no DEA model can include all the potential variables (can lead to partial and potentially misleading results), and standard DEA models do not consider stochastic variability in the data (susceptible to data errors).^39,47,57

There are two approaches to DEA models, the first approach is classically known as VRS (vector return to scale, or Banker, Charnes and Cooper), which considers variable returns to scale and does not assume proportionality between inputs and outputs; and the second approach, the CRS (constant return to scale, or Charnes, Cooper and Rhodes scale -), which considers constant scale returns and that any variation in inputs leads to a proportional variation in products.³⁶ As we aim to assess efficiency under management perspective, where the extent to which the scale of operations or different practices and primary care providers’ styles affect efficiency in PHC delivery, we opted for the VRS assumption.^33,35

The other approach to be considering in DEA refers to the efficiency analysis orientation, which typically can be input-oriented or output-oriented. In input-oriented DEA, the linear programming model determines how much the input use of a DMU could contract when used efficiently to achieve the same levels of output. In this sense, the input-oriented DEA is less relevant for estimating capacity utilization. On the other hand, in the output-oriented DEA, the linear programming model determines the DMU’s potential output given its levels of inputs, when operating efficiently (along with the best practice frontier). In other words, the output-oriented approach estimates the potential output for a given level of inputs and can be used to measure capacity utilization by means of the ratio between the actual to potential outputs. In the context of this study, the definition of the efficiency analysis orientation should be based upon what primary care managers are able to control better, either their already limited resources or health outcomes.⁵²

In fact, under a moral perspective, healthcare facilities should not aim to reduce inputs and costs but promote priority-setting and thus concentrate resources on improving health outcomes (outputs). Therefore, output-oriented is the model that best fits the perspective of this study, in which an ACES is made efficient through the proportional increase of the outputs while keeping its inputs unchanged. Nevertheless, as the output considered are avoidable diabetes hospitalization rates, the models included the inverse of the rates as outputs, in line with an approach conducted by Sahin and Ozcan.⁵³ This procedure was applied because the objective of the ACES should be to decrease the negative outcome (preventable hospitalizations) and not to reduce the human resources allocated at this level of care, given the scarcity of resources and the shortage of professionals in the health sector, as discussed by several authors.^58,59 Thus, it is intended that, with the same level of human resources in each ACES, better health indicators could be produced for the population (lower rates of preventable hospitalizations due to diabetes complications).

The input variables chosen to represent the models reflect human resources present in each ACES to provide care to their respective area of intervention. These input variables were represented by the number of medical doctors, number of resident doctors and number of nurses. The output variables to be included were the inverse of five PQI related to preventable diabetes hospitalizations²³: PQI 01 - Diabetes Short-Term Complications Admission Rate; PQI 03 - Diabetes Long-Term Complications Admission Rate; PQI 14 - Uncontrolled Diabetes Admission Rate; PQI 16 - Lower-Extremity Amputation Among Patients with Diabetes Rate: and PQI 93 - Prevention Quality Diabetes Composite. We considered separated models to assess each individual PQI and the composite indicator as each PQI reflects a distinct condition related to diabetes, despite possible correlations between them. Such conditions may differ in terms of prevalence and severity. In this sense, a primary care provider that performs well for a given condition may not perform equally well for another. Thereby, evaluating PHC efficiency for individual PQIs was the most appropriate approach to provide results that are clinically relevant and to identify which diabetes-related complications or conditions need to be addressed more urgently at primary care level.

Finally, as we considered the inverse of the PQI rates as output to the DEA models, some unbalanced scale issues affecting DEA estimates may arise as the resulting outputs assumed values that were lower than 1, contrasting with the input variables, which are counts of healthcare professionals. Therefore, we performed mean normalization to address this problem, as suggested in appropriate literature,⁶⁰ which can be described with the following general equation:

vNORMai=vaiv¯i

where, vNORM_ai is the normalized value v¯i of the input or output variable iassociated with DMU a, is the mean value of input or output i for the 54 ACES and v_ai represents the observed value of the input or output i for ACES a.

Tobit Regression

Tobit regression was performed using the efficiency scores obtained with the application of DEA as dependent variable to investigate the several factors influencing inefficiency in PHC. The tobit model is adopted when the dependent variable is observed only within a specific numerical range caused by a form of censorship in the observations. In this case, the estimation by ordinary least squares (OLS) does result in inconsistent estimates of the parameters, since the restricted range of the dependent variable can make the censored sample not representative of the population.⁶¹ Moreover, efficiency scores obtained with DEA consist in a relative rather than an absolute index, and thereby are correlated between them, making the OLS regression invalid.⁶² Therefore, Tobit regression, which is one of few dependent variable models addressing a censored structure,³⁴ is performed in this study using the maximum likelihood estimation method for parameter estimation.

It intended to discover which factors explain a larger proportion of the variability of the efficiency scores controlling for external factors in each ACES. The factors that could potentially influence efficiency in diabetes care were: Ratio of patients by medical doctors, Prevalence of DM, Proportion of elderly patients and unemployment rate, Proportion of patients with DM and record of HbA1c in the last six months, Proportion of patients with DM and with last HbA1c ≤8.0%, Proportion of patients with DM <65 years old with a HbA1c test ≤6.5%, Proportion of patients with DM and with microalbuminuria test in the last year, Proportion of patients with a risk assessment for diabetes type 2 (3 years), Adequate follow-up index of patients with DM, Cost of therapy for diabetic patients, and Cost of therapy for diabetic patients under control.

Results

Data Envelopment Analysis Model

A total of five DEA models were estimated for cross-sectional data of both years (2016 and 2017), each of which considering the five distinct PQIs related to diabetes as outputs. Furthermore, Tobit regression was applied in the second stage to identify the determinants (external or environmental factors) affecting the efficiency scores obtained in the first stage. Table 1 presents the descriptive statistics for inputs, outputs, and environmental variables of the 54 Portuguese health centre groupings (ACES) included in our sample.

In the first stage, basic radial DEA models were run to estimate efficiency, which consists of computing the ratio between outputs and inputs and comparing it with a benchmark. Fully efficient ACES are those with an efficiency score equal to 1, whereas values above 1 represent inefficiencies in the utilization of human resources to produce low PQI rates. Table 2 shows the mean efficiency scores for each PQI and year, considering the five output-oriented DEA models (one for each PQI) under VRS assumption. A total of 13 ACES reached the efficiency frontier in at least one PQI in a given year (Supplementary file 1, Table S1). The rate of ACES with efficiency scores above the MES in 2016 was 40.7, 46.3, 38.9, 46.3 and 51.9% for PQIs 01, 03, 14, 16 and 93, respectively. In 2017, this rate was found to be 50.0, 51.9, 44.4, 40.7 and 46.3% for PQIs 01, 03, 14, 16 and 93, respectively. The individual efficiency scores per ACES can be found in Supplementary file 1.

Considering the VRS assumption, an investigation into the input slacks for PQI 93 in 2016 revealed slacks in the number of doctor and nurses for 49 ACES, whereas input slacks in the number of intern doctors were identified for 44 ACES in 2016. In the following year, input slacks for doctors, nurses and intern doctors were found for 48, 35 and 40 ACES, respectively. Table 3 shows the input slacks averages for each year when using PQI 93 as output. The highest slack values were found for doctors and nurses, even though a considerable improvement was observed in 2017 for all professionals. Overall, these results indicate that most of the ACES have a substantial slack capacity in their health workforce.

Assuming the efficiency scores obtained with each DEA model as dependent variables, tobit regression censored at 1 was performed to assess the influence of environmental determinants on PHC efficiency in diabetes care. The estimation results of the Tobit fixed effects are presented in Table 4. The interpretation of the coefficients is similar to the OLS regression ie, how much the efficiency score can be increased or reduced following the one-unit increase of the independent variable. Residual analysis suggests good fit the models (Supplementary file 1). Five models were estimated considering the efficiency scores obtained for each PQI in 2017. Environmental variables for 2016 were included in the models as environmental determinants, as a lag of one year was assumed to examine the effects of such determinants on efficiency in diabetes care. A positive and statistically significant relation between efficiency and proportion of patients with DM <65 years old with a HbA1c test ≤6.5% was found for PQIs 03, 16 and 93, indicating that such control is important for improving efficiency scores (positive relation indicates decrease in the efficiency scores). On the other hand, a negative and statistically significant relation was found between efficiency and prevalence of DM and unemployment rate for PQI 1 and PQI 16, respectively. Moreover, a positive and statistically significant relation with efficiency was also found for proportion of elderly patients for PQIs 01.

When assessing the potential of improvements considering the fully efficient ACES as references for allocating human resources, a substantial improvement (reduction) in standardized PQI rates can be verified according to the models. These values represent optimal standardized PQI rates in a scenario in which inefficient ACES are placed in the efficiency frontier considering the combination of input and output values of the fully efficient ACES. Table 5 indicates, for each PQI, the target values for the output variable, represented by the optimal values of standardized PQI rates in 2016 and 2017. Regarding the composite indicator (PQI 93), the model estimated a reduction from an average of 82.8 to 43.3 hospitalizations per 100 000 inhabitants (reduction of 47.3%) in 2016, and from 66.2 to 35.6 hospitalizations per 100 000 inhabitants (reduction of 46.2%) in 2017. The remaining specific PQIs related to diabetes could also be reduced by a wide margin in both 2016 (PQI 01: -62.6%; PQI 03: -49.3%; PQI 14: -81.0%; PQI 16: -55.0%) and 2017 (PQI 01: -65.1%; PQI 03: -66.4%; PQI 14: -82.5%; PQI 16: -30.5%). The potential of improvement in the overall composite PQI (PQI 93) that can be achieved by each ACES can be found in Supplementary file 1 (Table S2).

Discussion

Primary care professionals’ activities include several different services, ranging from disease prevention, drug prescription, visits, and management of chronic diseases. In this study, we assessed primary care practices in managing diabetic patients. Several articles addressed the method for analyzing diabetes.^43,47 We propose a novel and different approach of evaluating the quality indicators in PHC for diabetes in an integrated manner with the assessment of preventable hospitalizations. In particular, the study evaluates the way different practices deliver care to patients affected by this chronic disease.

The production process considered in our study emphasizes the ideal use of human resources within ACES to produce better outcomes related to diabetes care, here summarized by avoidable hospitalization rates (PQIs). This efficiency assessment focusing on human resources is particularly relevant in the Portuguese health system, where gatekeeping is adopted and thus primary care providers, especially general practitioners, coordinate referrals to hospitals and specialists, and where a model of primary care unit called Family Health Units was introduced to promote group work as the main mode of provision of care, characterized by a higher cooperation between doctors, nurses and other PHC professionals.^63,64

Overall, assessing the DEA efficiency scores shows that improvement is still needed and possible for most of the ACES. In the first stage, it was found that most of the ACES were considered inefficient in different degrees. Individual ACES also presented great variability in efficiency according to the PQI used as output variable. The overall average efficiency scores worsened for PQI 01, 03 and 93 between 2016 and 2017. Furthermore, ACES performed worse for uncontrolled diabetes and lower-extremity amputation (PQIs 14 and 16), although these were the only indicators in which the overall mean efficiency improved in 2017.

An important policy implication of this study could be that the technically inefficient ACES can, on average, improve their current diabetes related PQI rates by around 50% according to the VRS models. Differences regarding the effect of the scale assumption on the measurement of capacity utilization, however, have a theoretical background and should be considered. The CRS model reflects the fact that outputs will change by the same proportion as inputs are changed, whereas VRS models reflect production process or technologies that may exhibit increasing, constant and decreasing returns to scale. The efficiency frontier defines the full capacity output, in this case by means of PQI standardized rates, given a fixed level of inputs. The assumption of a CRS frontier is likely to result in a greater estimate of capacity output in comparison with VRS frontier.⁶⁵ In the context of our study, however, VRS hypothesis seems to be more reliable, as it typically considers the effect on the relation between inputs and outputs and are more suitable when input or output variables are defined using ratios.⁵¹ In this sense, VRS frontier should better represent the production process involved in DM care at PHC level.

According to inputs slack analysis, underutilization of human resources capacity is observed in most of the ACES. It is important in terms of performance to use the slack capacity pointed in the DEA models, particularly in the number of doctors and nurses, which registered considerably high slack values under the VRS assumption. It is important to notice that this study only used human resources, but there are other factors that may influence avoidable hospitalizations in diabetes. On a more macroscopic perspective, the results could be impacted by a variety of contextual factors intrinsic to each ACES, which represents distinct regions with particular characteristics, including socio-economic determinants, demographics, behavior patterns (smoking, diet, alcohol consumption, physical activity, obesity, etc)⁶⁶ and burden of diseases. Nevertheless, in this study, we attempted to address this issue by including variables in the Tobit regression addressing regional characteristics, such as the proportion of elderly, burden of diseases (eg, prevalence of DM), socio-economic determinants such as unemployment rate, which is also proxy for income level, as well as several indicators regarding the management of diabetes at ACES level.

The results showed, however, that only a few studied external factors may influence the efficiency of ACES with regard to avoidable hospitalizations related to DM, in which the proportion of patients with DM <65 years old with a HbA1c test ≤6.5% indicator was by far the most relevant factor, affecting efficiency in a positive way for most PQIs, and which corroborates with the results obtained in previous integrated analyses developed by the authors,⁶⁷ apart from PQIs 1 and 14. Unemployment rate and prevalence of diabetes were the only determinants with a negative and statistically significant effect on efficiency, found for PQIs 1 and 16. The most surprising results, however, comes from the effects found for proportion of elderly, where an apparent positive and statistically significant effect on efficiency related to PQI 1 exists, which seems to contradict the conventional notion that increasing age would result in higher rates of avoidable hospitalization and thus would contribute to lower efficiency.

In general, the estimated DEA efficiency scores can provide insights into performance in primary care practices, helping decision makers to detect problems and plan appropriate strategies to improve and adequate case by case based upon best practices (efficient providers). Primary care will continue to play a critical role in the next years to address current challenges and thus provide higher quality of life to the Portuguese population. This assessment is particularly timely as demographics and burden of chronic diseases are increasing the health needs. Furthermore, it provides evidence on potential of improvement and how the current set of primary care professionals’ activities can be used to reduce avoidable hospitalizations, promoting thereby a shift in resources from hospitals to primary care.^68,69 Moreover, by using output-oriented models, it is possible to estimate the amount by which PQI rates could be improved (decreased) without enlarging the human resources. Some limitations, however, should be pointed in this study. A limitation of the DEA methodology relies on its deterministic nature, where the results are entirely dependent on the numeric values of inputs and outputs variables in the dataset. As the DEA model compares different DMUs to estimate relative efficiencies, the number and nature of DMUs present in the data can considerably change the results.⁷⁰ For instance, if a more efficient DMU is added to the dataset, the frontier would be moved, and some of the efficiency scores of other DMUs would fall accordingly. Additionally, the use of a different set of input and output variables, such as other indicators related to diabetes as outputs or other input variables representing the professionals’ activities or resources (eg, number of PHC encounters) might have generated different conclusions. Another limitation is the comparability of ACES regarding certain dimensions of care, such as accessibility to primary care. Different ACES present distinct population density and size, resulting in important differences in the supply and demand of primary care services. An ACES serving much lower population density areas could be easily placed on the efficiency frontier for presenting a much lower number of patients and hospitalizations relatively to its peers. Another dimension that could be considered is continuity of care, which can be considered as the relationship between patients and primary care providers that goes beyond medical appointments and episodes of illnesses. This dimension can be included in the efficiency analysis by means of continuity indicators, such as size and mobility of patients between lists of general practitioners, distribution of the workload within the PHC professionals and communication with other healthcare sectors (referrals).⁷¹ Some primary data were not available at the time of conducting this study, such as the use of quality indicators applied to integrated management between the levels of care (primary, secondary, tertiary), quality of life indicators, environmental behaviour variables, and the composition of the medical team or other variables that reflect cooperation are essential factors. Therefore, the authors recommend an assessment as soon as possible. While it is essential to recognize that it is not possible to solve all these underlying issues, we attempted to minimize it by limiting the production process to primary care practices delivered by key primary care professionals and considering a second-stage analysis to understand the relevant factors contributing to efficiency. A sensitivity analysis considering variables that comprehensively reflect the complexity of the practices may be considered in the future, to overcome these limitations and to evaluate primary care efficiency in a broader scenario.

Conclusion

This study provided an empirical picture of the efficiency in managing diabetes at primary care level in 54 Portuguese health centre groupings (ACES). Results showed that inefficiency exists in the practices of most ACES, as well as the existence of a great variability among them with respect to efficiency scores. Inefficient ACES could substantially improve avoidable hospitalization rates due to diabetes using their current level human resources differently. Estimates showed that among the environmental determinants influencing the efficiency of the ACES in diabetes care, the proportion of patients with DM <65 years old with a HbA1c test result ≤6.5% correlates with efficiency in a positive way (that is, the higher the proportion of these patients, the lower the PQI rates are) for all PQIs, but PQIs 1 and 14. Our findings could be used for primary care managers and stakeholders to direct their attention to benchmarking their practices within their most efficient regional peers, or another comparative ACES, in order to understand their performance in a more detailed way. Furthermore, it narrows a gap in the literature in healthcare efficiency assessment, as there are very few disease-based DEA studies focusing on efficiency of healthcare. Our study suggests that efficiency assessment is a useful way to obtain insights into providers’ needs and potential of improvement in diabetes care at PHC level, and thus could be integrated into healthcare delivery performance assessments considering broader and integrated scopes.

Acknowledgements

The authors would like to thank the Central Authority for Health Services, for providing access to the PHC and hospital admission data. Also, the authors would like to thank the Project “1st.IndiQare - Quality indicators in primary health care: validation and implementation of quality indicators as an assessment and comparison tool,” financed by FEDER - European Fund - Regional Development Funds through the COMPETE 2020 - Operational Program for Competitiveness and Internationalization (POCI), and by Portuguese funds through FCT – Fundação para a Ciência e Tecnologia, for the support to carry out this study (POCI-01-0145-FEDER-030766).

Ethical issues

Data were provided by the Central Administration of the Health System, IP. (ACSS), an official organ of the Ministry of Health, through a cooperation protocol with the Faculty of Medicine, University of Porto and CINTESIS for research purposes.

Competing interests

Authors declare that they have no competing interests.

Authors’ contributions

Study conception by AR and AF. Study design and methodology by AR and JS. Material preparation and data collection by AR, and the analysis performed by AR and JS. Validation: PC, ML, PS, and AF. Writing – original draft by AR. Writing – review & editing: JS, PC, ML, PS, and AF. Supervision: PS and AF. Funding acquisition: Not applicable. All authors read and approved the final manuscript and are responsible for the correctness of the statements provided.

Supplementary files

References

1. World Health Organization (WHO). Thirteenth General Programme of Work (GPW13): Methods for Impact Measurement. Geneva: WHO; 2020.
2. Mackenbach JP, Karanikolos M, McKee M. The unequal health of Europeans: successes and failures of policies. Lancet 2013; 381(9872):1125-1134. doi: 10.1016/s0140-6736(12)62082-0 [Crossref] [ Google Scholar]
3. Kluge H. A new vision for WHO’s European Region: united action for better health. Lancet Public Health 2020; 5(3):e133-e134. doi: 10.1016/s2468-2667(20)30003-7 [Crossref] [ Google Scholar]
4. Bindman AB, Grumbach K, Osmond D, Vranizan K, Stewart AL. Primary care and receipt of preventive services. J Gen Intern Med 1996; 11(5):269-276. doi: 10.1007/bf02598266 [Crossref] [ Google Scholar]
5. Starfield B. New paradigms for quality in primary care. Br J Gen Pract 2001; 51(465):303-309. [ Google Scholar]
6. Bynum B. A history of chronic diseases. Lancet 2015; 385(9963):105-6. doi: 10.1016/s0140-6736(15)60007-1 [Crossref] [ Google Scholar]
7. Ramalho A, Castro P, Gonçalves-Pinho M. Primary health care quality indicators: an umbrella review. PLoS One 2019; 14(8):e0220888. doi: 10.1371/journal.pone.0220888 [Crossref] [ Google Scholar]
8. Donaldson MS. Measuring the Quality of Health Care. Washington, DC: National Academy Press; 1999.
9. Mainz J. Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care 2003; 15(6):523-530. doi: 10.1093/intqhc/mzg081 [Crossref] [ Google Scholar]
10. Di Cesare M. Global trends of chronic non-communicable diseases risk factors. Eur J Public Health 2019; 29(Suppl 4):ckz185-196. doi: 10.1093/eurpub/ckz185.196 [Crossref] [ Google Scholar]
11. King H, Aubert RE, Herman WH. Global burden of diabetes, 1995-2025: prevalence, numerical estimates, and projections. Diabetes Care 1998; 21(9):1414-1431. doi: 10.2337/diacare.21.9.1414 [Crossref] [ Google Scholar]
12. Purdy S, Griffin T, Salisbury C, Sharp D. Ambulatory care sensitive conditions: terminology and disease coding need to be more specific to aid policy makers and clinicians. Public Health 2009; 123(2):169-173. doi: 10.1016/j.puhe.2008.11.001 [Crossref] [ Google Scholar]
13. Organisation for Economic Co-operation and Development (OECD). Health at a Glance 2019. Paris: OECD; 2019. 10.1787/4dd50c09-en.
14. American Diabetes Association. Diagnosis and classification of diabetes mellitus. Diabetes Care 2010; 33(Suppl 1):S62-69. doi: 10.2337/dc10-S062 [Crossref] [ Google Scholar]
15. Or Z. Exploring the Effects of Health Care on Mortality Across OECD Countries. Paris: OECD; 2001. 10.1787/716472585704.
16. International Diabetes Federation (IDF). IDF Diabetes Atlas. Brussels, Belgium: IDF; 2019.
17. Institute for Health Metrics and Evaluation (IHME). GBD Compare. Seattle, WA: IHME, University of Washington; 2015.
18. Kharroubi AT, Darwish HM. Diabetes mellitus: the epidemic of the century. World J Diabetes 2015; 6(6):850-867. doi: 10.4239/wjd.v6.i6.850 [Crossref] [ Google Scholar]
19. WHO. Diabetes Country Profile 2016. Portugal: WHO; 2016.
20. Dictorate-General of Health (DGS). National Programme of the Prevention and Control of Diabetes. Lisbon, Portugal: DGS; 2008.
21. Sarmento J, Rocha JVM, Santana R. Defining ambulatory care sensitive conditions for adults in Portugal. BMC Health Serv Res 2020; 20(1):754. doi: 10.1186/s12913-020-05620-9 [Crossref] [ Google Scholar]
22. Seringa J, Marques AP, Moita B. Influence of diabetes on multiple admissions for ambulatory care sensitive conditions. Eur J Public Health 2018; 28(Suppl 4):cky214-157. doi: 10.1093/eurpub/cky214.157 [Crossref] [ Google Scholar]
23. Agency for Healthcare Research and Quality (AHRQ). QITM Version v6.0 Prevention Quality Indicators Technical Specifications Updates – Version 502 v2019 (ICD 10-CM/PCS). AHRQ; 2019.
24. Sarmento J, Alves C, Oliveira P, Sebastião R, Santana R. [Characterization and evolution of avoidable admissions in Portugal: the impact of two methodologic approaches]. Acta Med Port 2015; 28(5):590-600. doi: 10.20344/amp.6324 [Crossref] [ Google Scholar]
25. Pinto A, Santos JV, Souza J. Comparison and impact of four different methodologies for identification of ambulatory care sensitive conditions. Int J Environ Res Public Health 2020; 17(21):8121. doi: 10.3390/ijerph17218121 [Crossref] [ Google Scholar]
26. Ramalho A, Lobo M, Duarte L, Souza J, Santos P, Freitas A. Landscapes on prevention quality indicators: a spatial analysis of diabetes preventable hospitalizations in Portugal (2016-2017). Int J Environ Res Public Health 2020; 17(22):8387. doi: 10.3390/ijerph17228387 [Crossref] [ Google Scholar]
27. Vuik SI, Fontana G, Mayer E, Darzi A. Do hospitalisations for ambulatory care sensitive conditions reflect low access to primary care? an observational cohort study of primary care usage prior to hospitalisation. BMJ Open 2017; 7(8):e015704. doi: 10.1136/bmjopen-2016-015704 [Crossref] [ Google Scholar]
28. Helmer DA, Tseng CL, Rajan M. Can ambulatory care prevent hospitalization for metabolic decompensation?. Med Care 2008; 46(2):148-157. doi: 10.1097/MLR.0b013e31815b9d66 [Crossref] [ Google Scholar]
29. Warner EA, Ziboh AU. The effects of outpatient management on hospitalization for ambulatory care sensitive conditions associated with diabetes mellitus. South Med J 2008; 101(8):815-817. doi: 10.1097/SMJ.0b013e31817cf785 [Crossref] [ Google Scholar]
30. Donaldson MS. Measuring the Quality of Health Care: A Statement by The National Roundtable on Health Care Quality. Washington, DC: National Academies Press; 1999.
31. Institute of Medicine (US) Committee on Quality of Health Care in America. Improving the 21st-century Health Care System. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001.
32. Wolfe A. Institute of Medicine Report: crossing the quality chasm: a new health care system for the 21st century. Policy Polit Nurs Pract 2001; 2(3):233-235. doi: 10.1177/152715440100200312 [Crossref] [ Google Scholar]
33. Organisation for Economic Co-operation and Development (OECD). Scoping Paper on Health System Efficiency Measurement. OECD; 2016.
34. Samut PK, Cafrı R. Analysis of the efficiency determinants of health systems in OECD countries by DEA and panel tobit. Soc Indic Res 2016; 129(1):113-132. doi: 10.1007/s11205-015-1094-3 [Crossref] [ Google Scholar]
35. Pelone F, Kringos DS, Valerio L. The measurement of relative efficiency of general practice and the implications for policy makers. Health Policy 2012; 107(2-3):258-268. doi: 10.1016/j.healthpol.2012.05.005 [Crossref] [ Google Scholar]
36. Charnes A, Cooper WW, Rhodes E. Measuring the efficiency of decision making units. Eur J Oper Res 1978; 2(6):429-444. doi: 10.1016/0377-2217(78)90138-8 [Crossref] [ Google Scholar]
37. Ramírez-Valdivia MT, Maturana S, Salvo-Garrido S. A multiple stage approach for performance improvement of primary healthcare practice. J Med Syst 2011; 35(5):1015-1028. doi: 10.1007/s10916-010-9438-7 [Crossref] [ Google Scholar]
38. Deidda M, Lupiáñez-Villanueva F, Codagnone C, Maghiros I. Using data envelopment analysis to analyse the efficiency of primary care units. J Med Syst 2014; 38(10):122. doi: 10.1007/s10916-014-0122-1 [Crossref] [ Google Scholar]
39. Cantor VJM, Poh KL. Integrated analysis of healthcare efficiency: a systematic review. J Med Syst 2017; 42(1):8. doi: 10.1007/s10916-017-0848-7 [Crossref] [ Google Scholar]
40. Baker D, Klein R. Explaining outputs of primary health care: population and practice factors. BMJ 1991; 303(6796):225-229. doi: 10.1136/bmj.303.6796.225 [Crossref] [ Google Scholar]
41. Milliken O, Devlin RA, Barham V, Hogg W, Dahrouge S, Russell G. Comparative efficiency assessment of primary care service delivery models using data envelopment analysis. Can Public Policy 2011; 37(1):85-109. doi: 10.3138/cpp.37.1.85 [Crossref] [ Google Scholar]
42. Banker RD, Conrad RF, Strauss RP. A comparative application of data envelopment analysis and translog methods: an illustrative study of hospital production. Manage Sci 1986; 32(1):30-44. doi: 10.1287/mnsc.32.1.30 [Crossref] [ Google Scholar]
43. Salinas-Jiménez J, Smith P. Data envelopment analysis applied to quality in primary health care. Ann Oper Res 1996; 67(1):141-161. doi: 10.1007/bf02187027 [Crossref] [ Google Scholar]
44. Bates JM, Baines DL, Whynes DK. Assessing efficiency in general practice: an application of data envelopment analysis. Health Serv Manage Res 1998; 11(2):103-108. doi: 10.1177/095148489801100204 [Crossref] [ Google Scholar]
45. Stefko R, Gavurova B, Kocisova K. Healthcare efficiency assessment using DEA analysis in the Slovak Republic. Health Econ Rev 2018; 8(1):6. doi: 10.1186/s13561-018-0191-9 [Crossref] [ Google Scholar]
46. Kringos DS, Boerma WG, Hutchinson A, van der Zee J, Groenewegen PP. The breadth of primary care: a systematic literature review of its core dimensions. BMC Health Serv Res 2010; 10:65. doi: 10.1186/1472-6963-10-65 [Crossref] [ Google Scholar]
47. Amado CAF, Dyson RG. Exploring the use of DEA for formative evaluation in primary diabetes care: an application to compare English practices. J Oper Res Soc 2009; 60(11):1469-1482. doi: 10.1057/jors.2008.160 [Crossref] [ Google Scholar]
48. Borg S, Gerdtham UG, Eeg-Olofsson K, Palaszewski B, Gudbjörnsdottir S. Quality of life in chronic conditions using patient-reported measures and biomarkers: a DEA analysis in type 1 diabetes. Health Econ Rev 2019; 9(1):31. doi: 10.1186/s13561-019-0248-4 [Crossref] [ Google Scholar]
49. Testi A, Fareed N, Ozcan YA, Tanfani E. Assessment of physician performance for diabetes: a bias-corrected data envelopment analysis model. Qual Prim Care 2013; 21(6):345-357. [ Google Scholar]
50. Thorsen M, McGarvey R, Thorsen A. Diabetes management at community health centers: examining associations with patient and regional characteristics, efficiency, and staffing patterns. Soc Sci Med 2020; 255:113017. doi: 10.1016/j.socscimed.2020.113017 [Crossref] [ Google Scholar]
51. Zakowska I, Godycki-Cwirko M. Data envelopment analysis applications in primary health care: a systematic review. Fam Pract 2020; 37(2):147-153. doi: 10.1093/fampra/cmz057 [Crossref] [ Google Scholar]
52. Hollingsworth B. Measuring efficiency in health care: analytic techniques and health policy. Economica 2010; 77(305):205-256. doi: 10.1111/j.1468-0335.2009.00763.x [Crossref] [ Google Scholar]
53. Sahin I, Ozcan YA. Public sector hospital efficiency for provincial markets in Turkey. J Med Syst 2000; 24(6):307-320. doi: 10.1023/a:1005576009257 [Crossref] [ Google Scholar]
54. BI-CSP - Portuguese Primary Health Care Indicators Control Panel. https://bicsp.min-saude.pt/pt/investigacao/Paginas/Matrizindicadorescsp_publico.aspx?isdlg=1.
55. Dictorate-General of Health: Bilhete de Identidade dos Indicadores dos Cuidados de Saúde Primários Contratualização. 2017. https://www.sns.gov.pt/wp-content/uploads/2017/04/bilhete_identidade_indicadores_contratualizacao_2017.pdf.
56. Statistics Portugal. Instituto Nacional de Estatística - As pessoas: 2017. Lisboa: INE; 2019. https://www.ine.pt/xurl/pub/320467122.
57. Khezrimotlagh D, Chen Y. Decision Making and Performance Evaluation Using Data Envelopment Analysis. Switzerland: Springer; 2018. International Series in Operations Research and Management Science. Vol 269.
58. Aluttis C, Bishaw T, Frank MW. The workforce for health in a globalized context--global shortages and international migration. Glob Health Action 2014; 7:23611. doi: 10.3402/gha.v7.23611 [Crossref] [ Google Scholar]
59. Crisp N, Chen L. Global supply of health professionals. N Engl J Med 2014; 370(10):950-957. doi: 10.1056/NEJMra1111610 [Crossref] [ Google Scholar]
60. Avkiran NK. Productivity Analysis in the Service Sector with Data Envelopment Analysis. In book: Excellence in Research Australia (ERA) - Collection. Camira. 2002. UQ Business School, The University of Queensland QLD 4072, Australia; 2006. http://www.users.on.net/~necmi/financesite/DEA%20Book%203rd%20Edition%202006_AVKIRAN.pdf.
61. Cameron A, Trivedi P. Microeconometrics Using Stata. College Station, TX: Stata Press; 2010.
62. Atkinson SE, Wilson PW. Comparing mean efficiency and productivity scores from small samples: a bootstrap methodology. J Product Anal 1995; 6(2):137-152. doi: 10.1007/bf01073408 [Crossref] [ Google Scholar]
63. Organisation for Economic Co-operation and Development (OECD). OECD Reviews of Health Care Quality: Portugal 2015. OECD; 2015.
64. Sripa P, Hayhoe B, Garg P, Majeed A, Greenfield G. Impact of GP gatekeeping on quality of care, and health outcomes, use, and expenditure: a systematic review. Br J Gen Pract 2019; 69(682):e294-e303. doi: 10.3399/bjgp19X702209 [Crossref] [ Google Scholar]
65. Pascoe S, Kirkley J, Gráboval D, Morrison-Paul CJ. Measuring and Assessing Capacity in Fisheries. Rome, Italy: Food and Agriculture Organization (FAO); 2003.
66. Varabyova Y, Müller JM. The efficiency of health care production in OECD countries: a systematic review and meta-analysis of cross-country comparisons. Health Policy 2016; 120(3):252-263. doi: 10.1016/j.healthpol.2015.12.005 [Crossref] [ Google Scholar]
67. Ramalho A, Castro P, Lobo M, Souza J, Santos P, Freitas A. Integrated quality assessment for diabetes care in Portuguese primary health care using prevention quality indicators. Prim Care Diabetes 2021; 15(3):507-512. doi: 10.1016/j.pcd.2021.01.001 [Crossref] [ Google Scholar]
68. Sibbald B, McDonald R, Roland M. Shifting care from hospitals to the community: a review of the evidence on quality and efficiency. J Health Serv Res Policy 2007; 12(2):110-117. doi: 10.1258/135581907780279611 [Crossref] [ Google Scholar]
69. Winpenny EM, Miani C, Pitchforth E, King S, Roland M. Improving the effectiveness and efficiency of outpatient services: a scoping review of interventions at the primary-secondary care interface. J Health Serv Res Policy 2017; 22(1):53-64. doi: 10.1177/1355819616648982 [Crossref] [ Google Scholar]
70. Ahmed S, Hasan MZ, MacLennan M. Measuring the efficiency of health systems in Asia: a data envelopment analysis. BMJ Open 2019; 9(3):e022155. doi: 10.1136/bmjopen-2018-022155 [Crossref] [ Google Scholar]
71. Kringos DS, Boerma WG, Hutchinson A, Saltman RB. Building Primary Care in A Changing Europe. Copenhagen, Denmark: European Observatory on Health Systems and Policies; 2015.