Developing a National Set of Health Equity Indicators Using a Consensus Building Process

Background

Equity in the provision of care, regardless of gender, ethnicity, geography, and social or economic status, was defined by the Committee on Quality of Health Care in America as one of the six dimensions of quality of care.¹ Health equity implies that everyone should have a fair opportunity to attain their full health potential.² Since the publication of the Commission on Social Determinants of Health (SDH) report in 2008,³ the equity discourse has related to this theoretical framework that explains the underlying processes that underpin health inequities. The framework includes three core components: (1) the socioeconomic and political context; (2) structural determinants of health inequities, for example social class divisions; and (3) intermediary determinants of health – housing, physical work environment, nutrition and physical activity. The structural determinants operate through intermediary determinants of health to shape health outcomes. As an “action oriented” framework, it is designed to help policy-makers pinpoint where to intervene to most effectively tackle health inequities.⁴ Inequity in access to healthcare, education, decent work and living conditions, developed neighbourhoods, healthy communities, trust and a sense of belonging, are major contributors to health inequities.^3,5,6 Therefore, efforts to reduce health inequities must also take into account the effects of education, income, ethnicity and geography on health outcomes. Monitoring health equity is guided by values, including the basic human right to health and non-discrimination.⁵ Reducing health inequities is a moral duty of governments and healthcare organizations. Governments have set up national health-equity surveillance systems for routine monitoring of health inequity and the SDH, and to evaluate the impact of health equity policy and interventions. Many countries, like Canada, New Zealand, the United States and United Kingdom have implemented measurement systems, based on an agreed set of indicators, allowing monitoring of the impact of interventions on the reduction of inequities between disadvantaged groups and the rest of the population.^7-10

Israel is characterized by considerable income inequalities, as manifested by a high Gini coefficient, compared to the majority of Organisation for Economic Co-operation and Development (OECD) countries.¹¹ National expenditure on health and its percentage of the gross domestic product is considerably lower than the OECD average. Health expenditure from public sources as a share of total spending, is lower compared with the OECD average (64% and 71% respectively),¹² resulting in a growing burden of out-of-pocket expenditure on health services. This affects socio-economically disadvantaged households the most,¹³ especially those in peripheral areas. Israel is a multi-ethnic country, with Jews comprising 74%, Arabs, mostly Muslim – 21%, and others – 5% of the total Israeli population.¹⁴ Despite its small size, two regions (North and South) are regarded as the periphery of the country. Both comprise small urban and rural communities, lower access to services, with lower socioeconomic status (SES), higher unemployment rates and higher proportion of populations at-risk for inequity, such as immigrants and Arabs, compared with the central urban regions.¹⁵

Israel has a public healthcare system, framed by the National Health Insurance Law that mandates a defined, comprehensive basket of health services to all citizens. Services are provided by four not-for-profit health maintenance organizations (HMOs).¹³ Since 2004, a National Program for Quality Indicators in Community Healthcare provides policy-makers and citizens with information on the quality of community healthcare in Israel. This yearly report, based on data from more than 90% of all Israeli citizens, includes several dozen indicators in health promotion and care for non-communicable diseases.¹⁶

In Israel, academic and national bodies, such as the Central Bureau of Statistics, have documented inequities related to education, income and ethnicity.⁹ The Ministry of Health (MoH) publishes a yearly report on health inequities, since 2010.¹⁷ Considerable disparities have been consistently reported, for example in infant mortality, risk factors like smoking and obesity, non-communicable diseases and life expectancy, between the Arab and Jewish populations,^18,19 and income-related disparities in health services utilization and health outcomes.²⁰ Other population groups identified as at-risk for health inequities are ultra-Orthodox Jews, and immigrants from countries of the former Soviet Union and from Ethiopia.¹³ These yearly reports include analysis of regional inequities, highlighting the gaps between the center and the periphery.^15,21 Despite diverse efforts,^22–25 mainly by the MoH and the four HMOs, considerable health gaps still exist between communities, populations, and regions of the country.²⁶ These gaps, coupled with a high national level of poverty and widening social disparities,²⁷ led the MoH to define equity measurements as part of its strategic work plan. In 2018, the MoH decided on the development of a set of national equity indicators, to guide its strategy towards more equitable health outcomes and allow regular monitoring of progress in the reduction of inequity in both health outcomes and access to health services.

Social participation in decision-making is a key driver for the promotion of health equity.²⁸ In light of this understanding, a Delphi technique was utilized to shape a collaborative, multidisciplinary, and multi-sectoral process to choose the set of national health equity indicators.²⁹ Delphi is a widely used and accepted technique for systematically gathering and combining data from a group of respondents within their area of expertise, in order to arrive at an informed consensus on a complex problem.³⁰ The technique is designed to achieve a convergence of opinion on a specific real-world issue, using iterative rounds of grading with controlled feedback reports from the research team. Participants have the option to reconsider their initial grading in light of the group scoring. This technique has been widely used to reach agreement among experts and/or public representatives on various issues, such as health policy,³¹ patient reported outcome measures,³² as well as in selection of equity³³ and quality³⁴ indicators.

The objective of this manuscript was to describe the process, of reaching a consensus among experts and public representatives on a defined set of evidence-based national indicators, aimed to monitor equity, or lack thereof, in health services provision and health outcomes.

Methods

Study Design and Setting

The study was commissioned by the MoH, which recently defined the development of quantifiable equity measures as part of its strategic work plan. The research team included the MoH’s senior representative (SA), a social worker andequity promotion expert; experts in quality management (RWM, LV, physician and nurse), equity promotion (RWM) and health promotion (LV), quality measurement and methodology (PFB, a sociologist).

The Israeli National Institute for Health Policy Research provided both the finance and a steering committee to guide the research. This committee was composed of 10 senior members from the fields of health equity (4 members, one of them a professor of social work), economics (1), health policy (2) and quality measurement (3).

The steering committee met with the researchers in person (frontal meetings) for the purpose of designing and monitoring the process as well as for summarizing the outputs of the process and the lessons learnt, once the consultation ended.

The multi-stage study took place during January 2019-June 2020 and consisted of the following stages:

Stage A – Identification of Potential Equity Measures

A1. Exploring diverse data sources: Interviews were conducted with stakeholders and data holders such as the directors of national registries at the Israel Centre for Disease Control, to map all available sources of indicators. The guiding principles in choosing the stakeholders were the relevance of data managed in their organization to the health of the population, their ability to provide data, and sector diversity, so as to include education and welfare stakeholders, in addition to those from the healthcare sector. All interviews comprised a semi-formal dialogue including (1) presentation of the aim of the research and what kind of data is needed, (2) learning about data availability in the domain or organization under consideration, and (3) defining what kind of data is required. The interviews were conducted by one of the researchers (RWM), considering that an open, sincere dialogue was the best way to achieve cooperation from the stakeholders.

A2. Defining the characteristics of a “good” indicator for the measurement of equity: In view of the interview findings, and based on the relevant literature, potential indicators were searched for, based on the following characteristics: Indicators of good quality,^35,36 according to their significance to the Israeli healthcare system³⁷ (ie, that measures an issue of public health importance, based on the clinical meaningfulness of the domain under consideration, and the size of the population affected); and solid scientific basis (ie, widely-accepted evidence base); reliability (ie, data is less amenable to manipulation); feasibility of data retrieval (preferably computerized data that is routinely collected, rather than data that requires active collection). The researchers and the steering committee based their definition of a “good indicator” on their 15-year-experience gained in national quality measurement,³⁷ a program that two of the researchers (LV and RWM) actively participated in, and the head of the steering committee led for a decade. Historical data of at least three consecutive years was regarded as an optional requirement for a “good” equity indicator, allowing evaluation of whether the gap widened, narrowed or remained stable over time. In fact, none of the indicators for which there were time trends data, demonstrated narrowing of the gap over time.

A3. Defining the SDH to guide data analysis. The following determinants were chosen, based on international experience^7-9,38 and data availability: age, sex, ethnicity, religiosity, immigration, and SES. Data on each indicator were presented to participants as differences between population groups, including male vs female, Arabs vs Jews, older vs younger, secular vs ultra-Orthodox, low to high SES quartile, and geographical centre vs periphery. Whenever data on ethnicity, immigration, income and education were unavailable at the individual level, proxy variables were calculated from the socio-demographic characteristics of the residential neighbourhood (N~3000 inhabitants), according to home address. An example is the proxy estimation of sociodemographic characteristics for all patients on dialysis for end-stage kidney disease during 2014-2018. The encrypted ID number of each individual patient was matched with the Internal Affairs Ministry’s registry of addresses. Each home address was translated using geo-coding (turning descriptive locational data such as a postal address or a named location into an absolute geographic reference)³⁹ to a specific geographic statistical area, with its unique sociodemographic attributes. Ethnicity, religiosity and SES of the area were allocated to the individual patient.

Only indicators with a relative difference of at least 10% between the at-risk population and others were included. These cut-off points were guided by the experience acquired in the two largest Israeli HMOs, Clalit Healthcare Services⁴⁰ and Maccabi Healthcare Services.⁴¹

The output of this first stage was a list of 30 equity indicators for presentation to participants in the consensus process. This number of measures was compatible with the literature about on-line Delphi techniques.^32,42

Stage B: Agreement on the Criteria for Equity Indicator Rating

Based on the literature and on the experience gained in the extant Israeli National Program for Quality Indicators in Community Healthcare,^37,43 the researchers identified and evaluated diverse criteria that could guide the Delphi participants on the rating of the inequity measures. The steering committee helped define the final four criteria. The relative weight of each criterion was based on anonymous voting by the steering committee members (n = 10). Scoring was on a scale of 0-100 points and the final weight for each criterion was the average of 10 scores. The selected criteria were: public health importance - the extent to which inequity in the specific health domain affects public health, and the size of population affected (relative weight 0.37); gap characteristics– the magnitude of the gap, the number of social determinants that shape the gap, its trend over time (0.26); potential for change– the ability of actions taken by the healthcare, education or welfare systems, to significantly reduce this gap (0.27); public interest - the importance of the issue and unfairness of the gap, as perceived by the public (0.10).

Stage C: Designing the Consultation Framework

Participants – Three of the researchers (RWM, LV and SA) suggested a list of potential participants, based on their previous experience with experts from diverse and wide-ranging areas, each holding the relevant knowledge and information in their field of work or advocacy.⁴⁴ After discussing the list of candidates among the researchers and consulting the steering committee, a list of 77 potential participants was proposed, aiming to include as diverse a range as possible, in terms of multi-sectorial – persons affiliated with the health, welfare and education sectors; multi-level – including senior and middle management employees, as well as direct service providers from the field; and diversity in terms of age, gender, geographic location and ethnicity. Among the 75 participants were experts in health economics, a leader of health policy and innovation in one of Israel’s HMOs, a leading professor of social work, the head of policy planning at the MoH, all 4 representatives of equity promotion in the 4 Israeli HMOs, patient and public representatives from the Israeli Mental Health Association and the Society of Patients’ Rights in Israel. These representatives were chosen due to their involvement in equity issues and ability to use digital platforms. Participants were not offered any incentive for participation.

Of the 75 individuals who were willing to participate, 25 were from the academia (research institutes, universities); 23 were involved in healthcare provision (medium level management and healthcare professionals, both from community and hospital settings); 17 were affiliated with non-governmental organizations, both patient and public representatives; and 10 were affiliated with government ministries (Table 1).

Special efforts were made to choose participants from the geographic periphery and the Arab ethnic minority. Despite these efforts, the geographic periphery and the Arab minority were under-represented (13.3% participants from the periphery vs 31% in the population; and 9.3% Arab participants vs 21% of the population). Participation was anonymous. Each participant was recruited by a personal phone call from one of the researchers.

A three-round Delphi was chosen to reach a consensus over a set of indicators. To allow busy participants from the whole country to participate, an online process was undertaken, facilitated by Qualtrics survey software.⁴⁵ The digital process allowed participants to rate the indicators individually, in their own time. Each inequity indicator was presented in a similar way: title and definition; description of relative differences by SDH - gender, age, SES, ethnicity, residence in the geographic periphery, and immigration – depending on data availability (Table S1, Supplementary file 1). When presenting the indicators, we used language that could be understood by diverse audiences, including laypersons and non-health professionals. Additional tables and diagrams were available to participants for more detailed, raw data (eg, the percent of smokers among Arab and Jewish males is 39% and 21.6% respectively). For each measure, participants were asked to rate all four criteria on a 5-point Likert scale, “1” indicated “not important at all” and “5” indicated “very important.” Rating of all indicators was mandatory, and participants could not continue to the next indicator until they had completed the scoring of the previous one, ensuring that all participants submitted full scoring sets.

The Delphi took place between February and May 2020, with intervals of 2 weeks between the first and second rounds and 8 weeks between the second and the final (third) rounds. The latter was postponed due to the outbreak of coronavirus disease 2019 (COVID-19), which caused increased burden on healthcare staff and management and timetable constraints among most of the participants.

In the first Delphi round, participants were invited to suggest additional indicators that they perceived to be important, to be included in the following 2 rounds. At the beginning of the second and third rounds, participants were informed of the scoring results from the previous round. One automated reminder was sent to non-respondents 5-7 days after the beginning of each round, followed by a personal reminder via SMS, by the researcher who recruited the participant. A graphic description of the process is presented in Figure. The middle column (Digital Delphi Survey) describes the number of indicators in each of the rounds.

Close

Figure. Flowchart of the Delphi Process for Selecting Equity Indicators.

Statistical Analysis

Calculation of Inequities Presented to Participants

Data from diverse sources were translated and presented as relative difference measures, which express difference between a measure of interest (rate, percentage, mean or some other quantitative measure) and the chosen reference point.⁴⁶ Relative difference was calculated as rate ratios, comprising the incidence rate in one group divided by the incidence rate in a comparison group. For example, in 2017, the prevalence of obesity among Arab females was 27.4%, while prevalence among Jewish females was 15.6%. We calculated the rate ratio as 27.4/15.6 = 1.76.

Data were presented to participants as relative difference, since this is the most widely used measurement of inequities in Israel.^40,41 Using this familiar measure in the consultation could not only further participation but also enhance the chances of implementation of the consensual indicators.

Full data on the relative differences for each indicator are presented in Table S1. Differences are presented as rate ratios between ethnic groups, genders, regions, age groups, SES levels, and religious groups.

Agreement Between Participants

Interrater reliability of the total score of each of the measures was quantified using an intra-class correlation coefficient (ICC) model (2,k), namely two-way random effects, absolute agreement and single rates/measurement. ICC demonstrates agreement between raters. This method was previously used in similar studies.⁴⁴ Based on a 95% confidence interval of the ICC estimate, ICC values less than 0.5 are indicative of poor reliability, between 0.5 and 0.75 indicate moderate reliability, between 0.75 and 0.9 indicate good reliability, and values greater than 0.90 indicate excellent reliability.⁴⁷

Further analysis was conducted to estimate agreement between raters for each indicator, utilizing one-sample Kolmogorov-Smirnov normal test (Table S2).⁴⁸

Results

Creation of Inequity Indicators

In-depth semi-formal interviews with data managers in the preparation phase revealed that data can be collected and pooled from multiple sources to create inequity indicators.

Indicators by Key Domain

Thirty inequity indicators were presented to participants at the beginning of the process (presented in Table 2).

These indicators represented the following 5 domains: health promotion and disease prevention (12 indicators), acute and chronic morbidity (10), mortality and life expectancy (2), healthcare resources, accessibility and affordability (4), and education and employment (2). For a full list of indicators and selected relative inequities, see Table S1.

Response and Delphi Participation Rates

Of the 75 people who agreed to participate, 72 (96%) completed the first round. In the second and third rounds, 64 out of 72 (88.9%) and 55 out of 64 (85.9%), respectively, completed the survey. Thus, the overall Delphi response rate – 55 out of 75 people who agreed to participate (73.3%) completed the 3 rounds.

Indicator Rating

The right column of Figure describes the agreement of the participants within each of the rating rounds.

Table 2 presents the overall weighted score of each of the 30 indicators, for each of the 3 Delphi rounds.

The 10 leading indicators at the end of the third round, with the final score (in parentheses) were: diabetes care (4.09 on a 5-point-scale), childhood obesity (4.02), ability to afford care (4.02), adult obesity (3.92), distribution of healthcare personnel (3.91), fatal childhood injuries (3.89), cigarette smoking (3.79), infant mortality (3.79), access to psychotherapy (3.77) and distribution of hospital beds (3.74).

Analysis of the scores allocated to each of the four screening criteria (Table 3), revealed that public health importance was given the highest score across all 30 indicators (average score 4.02) while potential for change received the lowest scores (average score 3.32). The character of the gap and public interest received intermediate scores (3.43 and 3.44, respectively).

Agreement Between Participants’ Scores

The degree of agreement across the 3 rounds is presented in Figure. Overall agreement on the 30 indicators rated in the first round was excellent (0.937); overall agreement on the 15 indicators rated in the second round and the 20 indicators rated in the third round was moderate (0.637 and 0.610, respectively). Agreement between 55 participants who rated the top 10 indicators during the third Delphi round was 0.751, representing a good agreement.

Table S2 of Supplementary file 1 presents the difference of the average of all raters’ scores from the mean score given to each of the 30 indicators. Differences are very small, thus supporting the ICC estimate of good level of agreement.

Discussion

This study demonstrated how a diverse range of experts reached a consensus on the 10 most important national equity indicators. Clinical domains were represented by health risk factors (eg, cigarette smoking), and health outcomes (eg, infant mortality). System indicators were represented by healthcare resources (care accessibility and affordability). Three of the ten indicators were related to children, emphasizing the importance of early intervention to increase the likelihood of better health across the lifecourse.⁴⁹ Several of the indicators, such as childhood injury or obesity, require collaboration between the healthcare and other sectors to improve equity. In line with the current equity approach that adopts an intersectional perspective,⁵⁰ the selected indicators reflect inequity in multiple SDH – gender, ethnicity, religiosity, socio-economic status, education, income and geographic periphery – which often intersect, amplifying the burden of individual inequities.³ The fact that 40% of the selected indicators are related to system factors including access, affordability and resource distribution, points to a deep understanding of the contribution of healthcare services as well as socioeconomic conditions, both reflecting the layers of the SDH framework.⁵¹ During design of the process, the researchers hoped for reasonable compliance and envisioned an indicator set that would contribute to the inter-sectoral nature of health equity. The selection of indicators fulfilled and even went beyond those expectations.

The level of agreement between participants was good in the first and second rounds and moderate in the final round. When analysing agreement on the 10 selected indicators among the 55 participants who completed the final rating, good agreement (ICC = 0.751) was achieved.³⁰ Given the range of participants’ backgrounds, from different sectors, levels, regions, and ethnic groups, this level of consensus is encouraging for the future implementation of these indicators.⁵²

A Delphi process conducted to select indicators for evaluation of population health in Europe used a similar process,⁴⁴ consulting with a wide range of experts to reach consensus on population health indicators, across economic and social factors, lifestyle and health behaviours, healthcare services and health outcomes among other fields. Infant mortality was also chosen as an indicator, as in the currently described process, with agreement between raters (Scott’s Pi inter-rater reliability coefficient) of 0.59. Overall agreement through the entire rating process ranged from 0.32-0.62.⁴⁴ Another study used Delphi technique to reach a consensus on public health research priorities to address health inequities. Participants were receptive to the method and motivated to respond and the technique was found to be practical and effective in obtaining opinions from a wide range of experts. The most important research priorities were mental health, a healthy environment and health behaviours, two of which coincide with indicators chosen in the current study.⁵³

This process should be evaluated in light of anticipated possible barriers that could interfere with its successful completion: (1) Insufficient data to create an appropriate set of potential inequity indicators: some of the indicators were “created” de novo, by combining diverse data sources, as described in the Methods section, thus broadening the existing set of indicators that could be presented for rating. (2) Complexity of content that might deter participants without a background in health: A prerequisite to effectively achieving a consensus is that all participants have an equal opportunity to understand the issue under consideration and participate in a meaningful and fair way.²⁵ The fairness of the process, whereby every participant had an equal voice, contributed to high engagement and compliance throughout the process. This is especially important in light of the heterogeneity of the participants, allowing a variety of viewpoints on the issue of health inequity,⁵⁴ which is caused by multiple factors and requires multidisciplinary solutions. Indicators chosen through consensus of a broad range of people from different fields are more likely to be acceptable and ultimately implemented.⁵²

An online Delphi technique is especially challenging, as there is no opportunity for further explanations or dialogue, which could have been provided in a face-to-face Delphi format or other methodologies, bringing together experts in a workshop format.⁵⁵ The presentation of the indicators in a clear and precise manner and request to rate the indicators in an easy-to-answer, closed format facilitated participation.⁵⁶ The spontaneous feedback from participants and the high response rate indicated that the survey tool was suitable for a diverse audience.

Enhancing the response rate is frequently addressed when conducting a Delphi survey.⁵⁷ The very high response rate in the first two rounds and especially the high response in the last round, which took place during the first peak of the COVID-19 pandemic, may be seen as an indication of high commitment of the participants towards the consensus process. This may be attributed to several factors including personal engagement of the researchers with the participants in the initial contact and the reminders sent; to the fact that the process was commissioned by the MoH, thus enhancing the perception that the study results were more likely to be implemented and thus reduce inequities; to the invitation of participants based on their expertise in the topics under consideration, which may have contributed to participants’ perception of the process as meaningful and their contribution as relevant⁵⁸; and to the accessibility and flexibility that the online consultation allowed.

Analysis of the scores allocated to each of the four screening criteria indicates that participants allocated the lowest scores to the “potential for change” criterion, reflecting their awareness of the challenges of reducing health inequity, such as the need to incorporate social or policy components to interventions directed at individuals.⁵⁹ An example of the challenging journey to improve health equity is Michael Marmot’s statement, ten years after the landmark review on health inequalities in England, that “the situation has become worse.”⁶⁰ Moreover, the dynamics of the ratings between the rounds show that participants re-considered their own scoring in light of the feedback provided about the group’s rating. Most indicators were rated lower in successive rounds, compared with the first round. This might indicate that participants took the rating seriously as it approached the final round, and were aware of the difficulties implicit in reducing inequity in the domains and indicators presented in the process. However, participants did not make a significant change between rounds, with 8 of the 10 selected indicators being rated among the 10 leading indicators in the first round. The similarity between the rating rounds is compatible with the findings of another Delphi process.⁴⁴

The multi-level, multi-sectoral background of the participants brought diverse voices and perspectives to the consensus process, increasing the chance of selecting indicators that are acceptable to a wide audience. Effective intervention to increase health equity should be multi-sectoral, given the nature of health determinants.⁶¹ Involvement of stakeholders in the process, such as those responsible for reducing health equities in the four HMOs, might increase the chances of implementing this set of indicators to guide interventions aimed at reducing inequity.⁵²

Moreover, the active involvement of the regulating body – the MoH – in the research, facilitated collaborations during the preparation phase and also increased the likelihood of translation of the study outputs into measurable inequity reduction targets for the national healthcare system.

This study bears several limitations. A relatively long interval between the second and third rounds, due to the burden imposed on participants during the peak of the COVID-19 outbreak, could have drastically reduced response rate, but in fact response rate remained high. The impact of such a health crisis on participants’ perspectives on the issues under consideration is difficult to determine. Despite extensive review of data sources, the 30 inequity indicators presented for selection do not cover all inequities in the health system. Moreover, data on health inequities from national registries are usually not up to date, and are often available with two or more years of lag, while other sources may be more up to date. All the data presented in the research were the latest available. Only a few indicators suggested by participants were suitable for inclusion in the selection process and based on reliable data, although none were highly rated. Given the number and diversity of the 30 indicators, we believe that the process covered most of the indicators that were available and suitable to enter the process. While efforts were made to recruit a diverse panel of experts, including those from minority groups, and from the periphery, the Delphi process by nature engages with experts in various fields, and may therefore exclude those with a language barrier, low literacy, or lack of access to technology. We recommend the design of a platform that allows disadvantaged groups to participate and bring the voices of those who need it most. Each equity indicator was analyzed and presented as relative difference. While the use of “absolute difference” (or absolute inequality) could add important information, we wanted our study to reflect the widely used measurements of inequities in Israel to further enhance participation and the chances of implementation of the agreed upon indicators. Once the indicators are well-established, they can be additionally presented in absolute terms. Last, but not least, SDH were somewhat underrepresented among the proposed indicators and none were included in the 10 most highly rated equity indicators. The impact of SDH on health has become even more relevant during the COVID-19 pandemic, which most significantly affected people with poor housing and working conditions, lower health literacy and pre-existing co-morbidities that exposed them to higher risks.^62,63 The pandemic, which has exacerbated disparities based on SDH, may act as a catalyst to give SDH a more prominent place within the health system.

A challenging and critical stage in transforming the set of indicators from “a grocery list” to a compass pointing in the “right direction” is the implementation stage. The MoH – who initiated the process – is accountable for adopting the indicator set and using it to guide governmental decision-making and inter-ministerial strategies. More specifically, the MoH has committed to translate the general definition of each of the 10 equity indicators into detailed operative definitions and to design dedicated computerized infrastructures to present these indicators on the MoH public site. Healthcare providers will be encouraged to use the indicators to guide their policy while the MoH will monitor national and regional progress towards reduction in inequities. Budgets will be allocated to help reduce regional inequities in healthcare infrastructures and health inequities between the least and the most deprived regions. Governmental policies, standards and regulations will reflect these strategies; ongoing monitoring and public reporting of these measures should be coupled with the setting of targets for the reduction of the highlighted inequalities.

Conclusion

The Delphi technique provided an easy to use, fair, and relatively quick method for consulting with a variety of experts and public representatives to choose a set of national equity indicators.

Policy-makers in countries which have not yet set a unified indicator set or those who wish to renew or expand an existing set can utilize the framework described here to build their own process, taking advantage of its strengths including evidence-based data, participatory nature, and low investment of time and resources.

The diverse set of indicators selected in this research could promote inter-sectoral collaboration both at the level of ministries and in the field. We hope that the development of a national set of equity indicators will produce a ripple effect, creating additional processes that will gradually expand public and professional involvement in the quest towards a more equitable healthcare system.

Ethical issues

The study protocol was approved by the Institutional Ethics Committee (Approval number 10370984).

Competing interests

Authors declare that they have no competing interests.

Authors’ contributions

All authors interpreted the data and edited and approved the final article. RWM, PFB, SA, LV, GP, MS, and OL conceptualized and designed the study, drafted the initial manuscript, and reviewed and revised the manuscript. AZ, VM, MS, and RWM designed the methods section, analyzed the data and reviewed and revised the manuscript. RWM and PFB critically reviewed the manuscript for important intellectual content.

Funding

The research was funded by a grant from The Israel National Institute for Health Policy Research (Grant number 0601445671).

Availability of data and materials

All data generated or analysed during this study are included in this published article.

Supplementary files

References

1. Institute of Medicine (US) Committee on Quality of Health Care in America.Crossing the Quality Chasm: A New Health System for the 21st Century. Washington (DC): National Academies Press (US); 2001. https://www.ncbi.nlm.nih.gov/books/NBK222278/. Accessed August 6, 2020.
2. Whitehead M. The concepts and principles of equity and health. Health Promot Int 1991; 6(3):217-228. doi: 10.1093/heapro/6.3.217 [Crossref] [ Google Scholar]
3. Marmot M, Friel S, Bell R, Houweling TA, Taylor S. Closing the gap in a generation: health equity through action on the social determinants of health. Lancet 2008; 372(9650):1661-1669. doi: 10.1016/s0140-6736(08)61690-6 [Crossref] [ Google Scholar]
4. Solar O, Irwin A. A Conceptual Framework for Action on the Social Determinants of Health.; 2010. https://drum.lib.umd.edu/handle/1903/23135. Accessed February 4, 2021.
5. Braveman P. Health disparities and health equity: concepts and measurement. Annu Rev Public Health 2006; 27:167-194. doi: 10.1146/annurev.publhealth.27.021405.102103 [Crossref] [ Google Scholar]
6. Healthy Prosperous Lives for All: The European Health Equity Status Report, 2019. https://www.euro.who.int/en/publications/abstracts/health-equity-status-report-2019.
7. Nixon SA, Lee K, Bhutta ZA. Canada’s global health role: supporting equity and global citizenship as a middle power. Lancet 2018; 391(10131):1736-1748. doi: 10.1016/s0140-6736(18)30322-2 [Crossref] [ Google Scholar]
8. Atkinson J, Salmond C, Crampton P. NZDep2013 Index of Deprivation. Wellington: Department of Public Health, University of Otago; 2014.
9. Abu-Saad K, Avni S, Kalter-Leibovici O. Health disparities monitoring in the US: lessons for monitoring efforts in Israel and other countries. Isr J Health Policy Res 2018; 7(1):14. doi: 10.1186/s13584-018-0208-1 [Crossref] [ Google Scholar]
10. Measuring inequalities in the UK for the Sustainable Development Goals - Office for National Statistics. https://www.ons.gov.uk/economy/nationalaccounts/uksectoraccounts/compendium/economicreview/july2018/measuringinequalitiesintheukforthesustainabledevelopmentgoals. Accessed August 20, 2020.
11. Income Inequality. https://www.oecd-ilibrary.org/social-issues-migration-health/income-inequality/indicator/english_459aa7f1-en. Accessed February 4, 2021. Published 2021.
12. OECD. Health at a Glance. Paris: OECD; 2019. 10.1787/4dd50c09-en.
13. Clarfield AM, Manor O, Nun GB. Health and health care in Israel: an introduction. Lancet 2017; 389(10088):2503-2513. doi: 10.1016/s0140-6736(17)30636-0 [Crossref] [ Google Scholar]
14. Population of Israel on the Eve of 2021.; 2021. https://www.cbs.gov.il/en/mediarelease/Pages/2020/Population-of-Israel-on-the-Eve-of-2021.aspx. Accessed February 4, 2021.
15. State of Israel. Gaps between Center and Periphery: Selected Data from the Society in Israel Report. State of Israel; 2019.
16. Jaffe A, Giveon S, Wulffhart L. Adult Arabs have higher risk for diabetes mellitus than Jews in Israel. PLoS One 2017; 12(5):e0176661. doi: 10.1371/journal.pone.0176661 [Crossref] [ Google Scholar]
17. Health inequality and coping with it, 2019. The Ministry of Health, Jerusalem 2020. https://www.health.gov.il/publicationsfiles/inequality-2019.pdf. Accessed June 8, 2021.
18. Chernichovsky D, Bisharat B, Brill A, Sharony C. The Health of the Arab Israeli Population. Taub Center report; 2017.
19. Muhsen K, Green MS, Soskolne V, Neumark Y. Inequalities in non-communicable diseases between the major population groups in Israel: achievements and challenges. Lancet 2017; 389(10088):2531-2541. doi: 10.1016/s0140-6736(17)30574-3 [Crossref] [ Google Scholar]
20. Shmueli A. Income-related inequalities in health and health services use in Israel. Isr J Health Policy Res 2014; 3:37. doi: 10.1186/2045-4015-3-37 [Crossref] [ Google Scholar]
21. Ministry of Health. Availability of community-based healthcare services in Israel: A spatial mapping. https://www.health.gov.il/PublicationsFiles/availability_03122017.pdf.
22. Balicer RD, Hoshen M, Cohen-Stavi C. Sustained reduction in health disparities achieved through targeted quality improvement: one-year follow-up on a three-year intervention. Health Serv Res 2015; 50(6):1891-1909. doi: 10.1111/1475-6773.12300 [Crossref] [ Google Scholar]
23. Wilf-Miron R, Galai N, Gabali A. Organisational efforts to improve quality while reducing healthcare disparities: the case of breast cancer screening among Arab women in Israel. Qual Saf Health Care 2010; 19(5):e36. doi: 10.1136/qshc.2008.029645 [Crossref] [ Google Scholar]
24. Horev T, Avni S. Strengthening the capacities of a national health authority in the effort to mitigate health inequity-the Israeli model. Isr J Health Policy Res 2016; 5:19. doi: 10.1186/s13584-016-0077-4 [Crossref] [ Google Scholar]
25. Theory NG-C, 2007 undefined. Bringing the mountain to the public: Dilemmas and contradictions in the procedures of public deliberation initiatives that aim to get “ordinary citizens” to deliberate. https://academic.oup.com/ct/article-abstract/17/4/411/4098783. Accessed July 9, 2020.
26. Skorecki K, Horton R. Israel: health and beyond. Lancet 2017; 389:2551-2562. doi: 10.1016/s0140-6736(17)30877-2 [Crossref] [ Google Scholar]
27. Bleikh H. Poverty and Inequality in Israel: Trends and Decompositions.; 2016. http://taubcenter.org.il/wp-content/files_mf/povertyandinequality.pdf. Accessed May 25, 2020.
28. Goldman S, Siman-Tov M, Bahouth H. The contribution of the Israeli trauma system to the survival of road traffic casualties. Traffic Inj Prev 2015; 16(4):368-373. doi: 10.1080/15389588.2014.940458 [Crossref] [ Google Scholar]
29. Landeta J, Barrutia J, Lertxundi A. Hybrid Delphi: a methodology to facilitate contribution from experts in professional contexts. Technol Forecast Soc Change 2011; 78(9):1629-1641. doi: 10.1016/j.techfore.2011.03.009 [Crossref] [ Google Scholar]
30. Seow H, Snyder CF, Mularski RA. A framework for assessing quality indicators for cancer care at the end of life. J Pain Symptom Manage 2009; 38(6):903-912. doi: 10.1016/j.jpainsymman.2009.04.024 [Crossref] [ Google Scholar]
31. Boylan S, Sainsbury E, Thow AM. A healthy, sustainable and safe food system: examining the perceptions and role of the Australian policy actor using a Delphi survey. Public Health Nutr 2019; 22(16):2921-2930. doi: 10.1017/s136898001900185x [Crossref] [ Google Scholar]
32. Lee J, Lee SH, Chang GT. Expert consensus on the development of a health-related questionnaire for the pediatric field of Korean medicine: a Delphi study. BMC Complement Med Ther 2020; 20(1):10. doi: 10.1186/s12906-019-2796-x [Crossref] [ Google Scholar]
33. Wong ST, Browne AJ, Varcoe C. Development of health equity indicators in primary health care organizations using a modified Delphi. PLoS One 2014; 9(12):e114563. doi: 10.1371/journal.pone.0114563 [Crossref] [ Google Scholar]
34. Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One 2011; 6(6):e20476. doi: 10.1371/journal.pone.0020476 [Crossref] [ Google Scholar]
35. Dimick J. What makes a “good” quality indicator?: Comment on “Quality indicators for evaluating trauma care.” https://jamanetwork.com/journals/jamasurgery/article-abstract/405767. Accessed February 4, 2021. Published 2010.
36. Blas E, Ataguba JE, Huda TM, Bao GK, Rasella D, Gerecke MR. The feasibility of measuring and monitoring social determinants of health and the relevance for policy and programme - a qualitative assessment of four countries. Glob Health Action 2016; 9:29002. doi: 10.3402/gha.v9.29002 [Crossref] [ Google Scholar]
37. Jaffe DH, Shmueli A, Ben-Yehuda A. Community healthcare in Israel: quality indicators 2007-2009. Isr J Health Policy Res 2012; 1(1):3. doi: 10.1186/2045-4015-1-3 [Crossref] [ Google Scholar]
38. Maharaj V, Rahman F, Adamson L. Tackling child health inequalities due to deprivation: using health equity audit to improve and monitor access to a community paediatric service. Child Care Health Dev 2014; 40(2):223-230. doi: 10.1111/cch.12011 [Crossref] [ Google Scholar]
39. Goldberg DW, Wilson JP, Knoblock CA. From text to geographic coordinates: the current state of geocoding. J Urban Reg Inf Syst Assoc 2007; 19(1):33-46. [ Google Scholar]
40. Balicer RD, Shadmi E, Lieberman N. Reducing health disparities: strategy planning and implementation in Israel’s largest health care organization. Health Serv Res 2011; 46(4):1281-1299. doi: 10.1111/j.1475-6773.2011.01247.x [Crossref] [ Google Scholar]
41. Wilf-Miron R, Tov OS, Lewinhoff I. [Equity promotion in Maccabi Healthcare Services: from the equality report to an organizational action plan]. Harefuah 2010; 149(4):210-213. [ Google Scholar]
42. Weiss-Laxer NS, Crandall A, Okano L, Riley AW. Building a foundation for family health measurement in national surveys: a modified Delphi expert process. Matern Child Health J 2020; 24(3):259-266. doi: 10.1007/s10995-019-02870-w [Crossref] [ Google Scholar]
43. Hsu CC, Sandford BA. The Delphi technique: making sense of consensus. Pract Assess Res Eval 2007; 12(1):10. doi: 10.7275/pdz9-th90 [Crossref] [ Google Scholar]
44. Freitas Â, Santana P, Oliveira MD, Almendra R, Bana ECJC, Bana ECCA. Indicators for evaluating European population health: a Delphi selection process. BMC Public Health 2018; 18(1):557. doi: 10.1186/s12889-018-5463-0 [Crossref] [ Google Scholar]
45. Snow J, Mann M. Qualtrics Survey Software: Handbook for Research Professionals. Qualtrics Labs, Inc; 2013.
46. Keppel K, Pamuk E, Lynch J, et al. Methodological issues in measuring health disparities. Vital Health Stat 2. 2005(141):1-16.
47. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15(2):155-163. doi: 10.1016/j.jcm.2016.02.012 [Crossref] [ Google Scholar]
48. Weiss MS. Modification of the Kolmogorov-Smirnov statistic for use with correlated data. J Am Stat Assoc 1978; 73(364):872-875. doi: 10.1080/01621459.1978.10480116 [Crossref] [ Google Scholar]
49. Chan M. Linking child survival and child development for health, equity, and sustainable development. Lancet 2013; 381(9877):1514-1515. doi: 10.1016/s0140-6736(13)60944-7 [Crossref] [ Google Scholar]
50. Green MA, Evans CR, Subramanian SV. Can intersectionality theory enrich population health research?. Soc Sci Med 2017; 178:214-216. doi: 10.1016/j.socscimed.2017.02.029 [Crossref] [ Google Scholar]
51. Bambra C, Gibson M, Sowden A, Wright K, Whitehead M, Petticrew M. Tackling the wider social determinants of health and health inequalities: evidence from systematic reviews. J Epidemiol Community Health 2010; 64(4):284-291. doi: 10.1136/jech.2008.082743 [Crossref] [ Google Scholar]
52. Fawcett S, Schultz J, Watson-Thompson J, Fox M, Bremby R. Building multisectoral partnerships for population health and health equity. Prev Chronic Dis 2010; 7(6):A118. [ Google Scholar]
53. Turner S, Ollerhead E, Cook A. Identifying research priorities for public health research to address health inequalities: use of Delphi-like survey methods. Health Res Policy Syst 2017; 15(1):87. doi: 10.1186/s12961-017-0252-2 [Crossref] [ Google Scholar]
54. Rowe G, Wright G. The Delphi technique: past, present, and future prospects—Introduction to the special issue. Technol Forecast Soc Change 2011; 78(9):1487-1490. doi: 10.1016/j.techfore.2011.09.002 [Crossref] [ Google Scholar]
55. Bobrovitz N, Parrilla JS, Santana M, Straus SE, Stelfox HT. A qualitative analysis of a consensus process to develop quality indicators of injury care. Implement Sci 2013; 8:45. doi: 10.1186/1748-5908-8-45 [Crossref] [ Google Scholar]
56. Frewer LJ, Fischer ARH, Wentholt MTA. The use of Delphi methodology in agrifood policy development: some lessons learned. Technol Forecast Soc Change 2011; 78(9):1514-1525. doi: 10.1016/j.techfore.2011.05.005 [Crossref] [ Google Scholar]
57. Keeney S, Hasson F, McKenna H. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs 2006; 53(2):205-212. doi: 10.1111/j.1365-2648.2006.03716.x [Crossref] [ Google Scholar]
58. Goluchowicz K, Blind K. Identification of future fields of standardisation: an explorative application of the Delphi methodology. Technol Forecast Soc Change 2011; 78(9):1526-1541. doi: 10.1016/j.techfore.2011.04.014 [Crossref] [ Google Scholar]
59. Lorenc T, Petticrew M, Welch V, Tugwell P. What types of interventions generate inequalities? Evidence from systematic reviews. J Epidemiol Community Health 2013; 67(2):190-193. doi: 10.1136/jech-2012-201257 [Crossref] [ Google Scholar]
60. Marmot M. Health equity in England: the Marmot review 10 years on. BMJ 2020; 368:m693. doi: 10.1136/bmj.m693 [Crossref] [ Google Scholar]
61. Arora M, Chauhan K, John S, Mukhopadhyay A. Multi-sectoral action for addressing social determinants of noncommunicable diseases and mainstreaming health promotion in national health programmes in India. Indian J Community Med 2011; 36(Suppl 1):S43-49. doi: 10.4103/0970-0218.94708 [Crossref] [ Google Scholar]
62. Horton R. Offline: COVID-19 is not a pandemic. Lancet 2020; 396(10255):874. doi: 10.1016/s0140-6736(20)32000-6 [Crossref] [ Google Scholar]
63. Paremoer L, Nandi S, Serag H, Baum F. Covid-19 pandemic and the social determinants of health. BMJ 2021; 372:n129. doi: 10.1136/bmj.n129 [Crossref] [ Google Scholar]