Games People Play: Lessons on Performance Measure Gaming from New Zealand; Comment on “Gaming New Zealand’s Emergency Department Target: How and Why Did It Vary Over Time and Between Organisations?”

Document Type : Commentary

Author

1 Center for Advanced Methods Development, RTI International, Durham, NC, USA

2 Population and Quantitative Health Sciences, University of Massachusetts Medical School, Worcester, MA, USA

Abstract

For decades, observers have noted that gaming of performance measurement appears to be both endemic and endlessly creative. A recent study by Tenbensel and colleagues provides a detailed look at gaming of a health system performance measure—emergency department (ED) wait time —within four hospitals in New Zealand. Combined, these four hospitals handled more than 25% of the ED visits in the country each year. Tenbensel and colleagues examine whether the New Zealand ED wait time target was set appropriately and whether we can trust any performance measure statistics that are not independently verified or audited. Their thought-provoking examination is relevant to anyone working in quality improvement and provides a valuable set of tools for detecting gaming in performance measurement.

Keywords

References

  1. Tenbensel T, Jones P, Chalmers L, Ameratunga S, Carswell P. Gaming New Zealand’s emergency department target: how and why did it vary over time and between organisations? Int J Health Policy Manag. 2020;9(4):152-162. doi:10.15171/ijhpm.2019.98
  2. Tenbensel T, Chalmers L, Jones P, Appleton-Dyer S, Walton L, Ameratunga S. New Zealand's emergency department target - did it reduce ED length of stay, and if so, how and when? BMC Health Serv Res. 2017;17(1):678. doi:10.1186/s12913-017-2617-1
  3. Ministry of Health NZ. How is My DHB Performing? https://www.health.govt.nz/new-zealand-health-system/health-targets/how-my-dhb-performing-2017-18.  Accessed January 29, 2020. Published 2020.
  4. Chalmers LM. Inside the Black Box of Emergency Department Time Target Implementation in New Zealand [dissertation]. New Zealand:  University of Auckland; 2014.
  5. Horwitz LI, Green J, Bradley EH. US emergency department performance on wait time and length of visit. Ann Emerg Med. 2010;55(2):133-141. doi:10.1016/j.annemergmed.2009.07.023
  6. Mason S, Weber EJ, Coster J, Freeman J, Locker T. Time patients spend in the emergency department: England's 4-hour rule-a case of hitting the target but missing the point? Ann Emerg Med. 2012;59(5):341-349. doi:10.1016/j.annemergmed.2011.08.017
  7. Howell E. The Key Findings Report for the 2008 Emergency Department Survey. Oxford: Picker Institute Europe; 2009.
  8. Mason S. Keynote address: United Kingdom experiences of evaluating performance and quality in emergency medicine. Acad Emerg Med. 2011;18(12):1234-1238. doi:10.1111/j.1553-2712.2011.01237.x
  9. Boyle A, Mason S. What has the 4-hour access standard achieved? Br J Hosp Med (Lond). 2014;75(11):620-622. doi:10.12968/hmed.2014.75.11.620
  10. Mannion R, Braithwaite J. Unintended consequences of performance measurement in healthcare: 20 salutary lessons from the English National Health Service. Intern Med J. 2012;42(5):569-574. doi:10.1111/j.1445-5994.2012.02766.x
  11. Ridgway VF. Dysfunctional consequences of performance measurements. Adm Sci Q. 1956;1(2):240-247. doi:10.2307/2390989
  12. Braithwaite J. Changing how we think about healthcare improvement. BMJ. 2018;361:k2014. doi:10.1136/bmj.k2014
  13. Feng Z, Wright B, Mor V. Sharp rise in Medicare enrollees being held in hospitals for observation raises concerns about causes and consequences. Health Aff (Millwood). 2012;31(6):1251-1259. doi:10.1377/hlthaff.2012.0129
  14. Wright B, O'Shea AM, Ayyagari P, Ugwi PG, Kaboli P, Vaughan Sarrazin M. Observation rates at veterans' hospitals more than doubled during 2005-13, similar to Medicare trends. Health Aff (Millwood). 2015;34(10):1730-1737. doi:10.1377/hlthaff.2014.1474
  15. Delia D, Cantor JC. Emergency department utilization and capacity. Synth Proj Res Synth Rep. 2009;(17):45929.
  16. Himmelstein D, Woolhandler S. Quality Improvement: ‘Become Good at Cheating and You Never Need to Become Good at Anything Else.’ Health Affairs Blog; 2015. doi:10.1377/hblog20150827.050132
  17. Martin GP, Wright B, Ahmed A, Banerjee J, Mason S, Roland D. Use or abuse? a qualitative study of emergency physicians' views on use of observation stays at three hospitals in the united states and england. Ann Emerg Med. 2017;69(3):284-292.e282. doi:10.1016/j.annemergmed.2016.08.458
  18. Wright B, Martin GP, Ahmed A, Banerjee J, Mason S, Roland D. How the availability of observation status affects emergency physician decisionmaking. Ann Emerg Med. 2018;72(4):401-409. doi:10.1016/j.annemergmed.2018.04.023
  19. Wright B, Zhang X, Rahman M, Kocher K. Informing Medicare's two-midnight rule policy with an analysis of hospital-based long observation stays. Ann Emerg Med. 2018;72(2):166-170. doi:10.1016/j.annemergmed.2018.02.005
International Journal of Health Policy and Management
Volume 10, Issue 4
April 2021
Pages 225-227

Files

History

  • Receive Date: 30 January 2020
  • Revise Date: 04 March 2020
  • Accept Date: 05 March 2020
  • First Publish Date: 01 April 2021

Share

How to cite

Statistics