Improve the Design and Implementation of Metrics From the Perspective of Complexity Science; Comment on “Gaming New Zealand’s Emergency Department Target: How and Why Did It Vary Over Time and Between Organisations?”

Document Type : Commentary

Author

The CareVoice, Shanghai, China

Abstract

From the perspective of complexity science, this commentary addresses Tenbensel and colleagues’ study, which reveals varied gaming behaviours to meet the New Zealand Emergency Department (ED) metric. Seven complexity-informed principles previously published in this Journal are applied to formulate recommendations to improve the design and implementation of metrics. (1) Acknowledge unpredictability. When designing a metric, policy-makers need to leave room for flexibility to account for unforeseen situations. When implementing a metric, they need to promote sense-making of relevant stakeholders. (2) Sense-making shall be encouraged because it is a social process to understand a metric, align values and develop a coherent strategy. Sense-making is important to (3) cope with self-organised gaming behaviours and to (4) facilitate interdependencies between ED and other departments as well as organisations. (5) We also need to attend to the relationship between senior management and frontline staff. Additionally, to address one of the methodological weaknesses in Tenbensel and colleagues’ study, (6) adaptive research approach is needed to better answer emerging questions. (7) Conflict should be harnessed productively. I hope these recommendations could limit gaming in future metrics and encourage stakeholders to view inevitable gaming as an improvement opportunity.

Keywords

References

  1. Tenbensel T, Jones P, Chalmers LM, Ameratunga S, Carswell P. Gaming New Zealand’s Emergency Department target: how and why did it vary over time and between organisations? Int J Health Policy Manag. 2020;9(4):152-162. doi:10.15171/IJHPM.2019.98
  2. Kitson A, Brook A, Harvey G, et al. Using Complexity and Network Concepts to Inform Healthcare Knowledge Translation. Int J Health Policy Manag. 2018;7(3):231-243. doi:10.15171/ijhpm.2017.79
  3. Grudniewicz A, Tenbensel T, Evans JM, Steele Gray C, Baker GR, Wodchis WP. ‘Complexity-compatible’ policy for integrated care? Lessons from the implementation of Ontario’s Health Links. Soc Sci Med. 2018;198:95-102. doi:10.1016/J.SOCSCIMED.2017.12.029
  4. Tenbensel T. Bridging complexity theory and hierarchies, markets, networks, communities: a ‘population genetics’ framework for understanding institutional change from within. Public Manag Rev. 2018;20(7):1032-1051. doi:10.1080/14719037.2017.1364409
  5. Hannigan B. Connections and consequences in complex systems: Insights from a case study of the emergence and local impact of crisis resolution and home treatment services. Soc Sci Med. 2013;93:212-219. doi:10.1016/j.socscimed.2011.12.044
  6. Churruca K, Pomare C, Ellis LA, Long JC, Braithwaite J. The influence of complexity: a bibliometric analysis of complexity science in healthcare. BMJ Open. 2019;9(3):e027308. doi:10.1136/bmjopen-2018-027308
  7. Greenhalgh T. Bridging the ‘two cultures’ of research and service: can complexity theory help? Comment on “experience of health leadership in partnering with university-based researchers in Canada – a call to ‘re-imagine’ research.” Int J Health Policy Manag. 2020;9(2):87-88. doi:10.15171/ijhpm.2019.89
  8. Long KM, McDermott F, Meadows GN. Being pragmatic about healthcare complexity: Our experiences applying complexity theory and pragmatism to health services research. BMC Med. 2018;16(1):94. doi:10.1186/s12916-018-1087-6
  9. Centers for Medicare & Medicaid Services. CMS Web Interface Measure Documentation. Centers for Medicare & Medicaid Services. https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/sharedsavingsprogram/Downloads/cms-web-interface-measure-documentation.pdf.  Published 2019. Accessed February 9, 2020.
  10. Centers for Medicare & Medicaid Services. Head CT or MRI Scan Results for Acute Ischemic Stroke or Hemorrhagic Stroke Patients who Received Head CT or MRI Scan Interpretation within 45 minutes of ED Arrival. National Quality Forum. https://cmit.cms.gov/CMIT_public/ViewMeasure?MeasureId=918.
  11. New Zealand Ministry of Health. Emergency department triage. New Zealand Government. https://www.health.govt.nz/our-work/hospitals-and-specialist-care/emergency-departments/emergency-department-triage. Accessed March 24, 2020. Published June 30, 2011.
  12. Hanefeld J, Powell-Jackson T, Balabanova D. Understanding and measuring quality of care: dealing with complexity. Bull World Health Organ. 2017;95:368-337. doi:10.2471/BLT.16.179309
  13. O’Connor T, Wong HY. Emergent Properties. In: Stanford Encyclopedia of Philosophy. Spring 202. Metaphysics Research Lab, Stanford University; 2015. https://plato.stanford.edu/archives/spr2020/entries/properties-emergent/.
  14. Ariely D. How honest people cheat. Harv Bus Rev. 2008;86(2):24.
  15. Geyer R. Can complexity move UK policy beyond ‘evidence-based policy making’ and the ‘audit culture’? Applying a ‘complexity cascade’ to education and health policy. Polit Stud. 2011;60(1):20-43. doi:10.1111/j.1467-9248.2011.00903.x
  16. Ardagh M. How to achieve New Zealand’s shorter stays in emergency departments health target. N Z Med J. 2010;123(1316):95-103.
  17. Jones P, Chalmers L, Wells S, et al. Implementing performance improvement in New Zealand emergency departments: the six hour time target policy national research project protocol. BMC Health Serv Res. 2012;12(1):45. doi:10.1186/1472-6963-12-45
  18. Gabbay J, le May A. Evidence based guidelines or collectively constructed “mindlines?” Ethnographic study of knowledge management in primary care. BMJ. 2004;329(7473):1013.
  19. Braithwaite J, Churruca K, Long JC, Ellis LA, Herkes J. When complexity science meets implementation science: A theoretical and empirical analysis of systems change. BMC Med. 2018;16(1). doi:10.1186/s12916-018-1057-z
  20. Greenfield D, Hinchcliff R, Banks M, et al. Analysing “big picture” policy reform mechanisms: The Australian health service safety and quality accreditation scheme. Health Expect. 2015;18(6):3110-3122. doi:10.1111/hex.12300
  21. Muller JZ. The Tyranny of Metrics. Princeton: Princeton University Press; 2018.
  22. Giuseppe M. Dealing with Fairness in Public Policy Analysis: A Methodological Framework. Luxembourg; Publications Office of the EU: 2017. doi:10.2760/75185
  23. Paley J, Eva G. Complexity theory as an approach to explanation in healthcare: A critical discussion. Int J Nurs Stud. 2011;48(2):269-279. doi:10.1016/j.ijnurstu.2010.09.012
  24. Yackinous WS. Overview of an Ecological System Dynamics Framework. In: Yackinous WS, ed. Understanding Complex Ecosystem Dynamics: A Systems and Engineering Perspective. Boston: Academic Press; 2015:83-91. doi:10.1016/B978-0-12-802031-9.00005-X
  25. Jiang D-Y, Lin Y-C, Lin L-C. Business Moral Values of Supervisors and Subordinates and Their Effect on Employee Effectiveness. J Bus Ethics. 2011;100(2):239-252.
  26. McCann L, Granter E, Hassard J, Hyde P. “You Can’t Do Both—Something Will Give”: Limitations of the Targets Culture in Managing UK Health Care Workforces. Hum Resour Manage. 2015;54(5):773-791. doi:10.1002/hrm.21701
  27. Harris M, Tayler B. Don’t Let Metrics Undermine Your Business. Harv Bus Rev. 2019;97(5):62-69.
  28. Nugus P, Carroll K, Hewett DG, Short A, Forero R, Braithwaite J. Integrated care in the emergency department: A complex adaptive systems perspective. Soc Sci Med. 2010;71(11):1997-2004. doi:10.1016/j.socscimed.2010.08.013
  29. Plsek PE, Wilson T. Complexity science: Complexity, leadership, and management in healthcare organisations. Br Med J. 2001;323(7315):746-749. doi:10.1136/bmj.323.7315.746
  30. Mark L. Managerialism and public attitudes towards UK NHS managers. J Manag Med. 1997;11(4):214-221. doi:10.1108/02689239710177314
  31. Faruk M. NHS managers’ views of their culture and their public image: The implications for NHS reforms. Int J Public Sect Manag. 2003;16(7):549-563. doi:10.1108/09513550310500409
  32. Garvin DA. How Google Sold Its Engineers on Management. Harv Bus Rev. 2013;91(12):74-82.
  33. Back J, Ross AJ, Duncan MD, Jaye P, Henderson K, Anderson JE. Emergency department escalation in theory and practice: a mixed-methods study using a model of organizational resilience. Ann Emerg Med. 2017;70(5):659-671. doi:10.1016/j.annemergmed.2017.04.032
  34. Greenhalgh T, Papoutsi C. Studying complexity in health services research: desperately seeking an overdue paradigm shift. BMC Med. 2018;16(1):95. doi:10.1186/s12916-018-1089-4
  35. Wisdom J, Creswell JW. Mixed Methods: Integrating Quantitative and Qualitative Data Collection and Analysis While Studying Patient-Centered Medical Home Models. Rockville, Maryland; 2013. https://pcmh.ahrq.gov/page/mixed-methods-integrating-quantitative-and-qualitative-data-collection-and-analysis-while.
  36. Engineering and Physical Sciences Research Council. Management of Large Research Activities and Centres. Swindon; 2006. https://epsrc.ukri.org/newsevents/pubs/management-of-large-research-activities-and-centres/.
  37. Plsek PE, Greenhalgh T. The challenge of complexity in health care. BMJ. 2001;323(7313):625. doi:10.1136/bmj.323.7313.625

Files

History

  • Receive Date: 09 February 2020
  • Revise Date: 21 March 2020
  • Accept Date: 24 March 2020
  • First Publish Date: 01 May 2021

Share

How to cite

Statistics