Measuring Hospital Performance Using Mortality Rates: An Alternative to the RAMR

Document Type : Original Article

Authors

College of Business, Stony Brook University, Stony Brook, NY, USA

Abstract

Background
The risk-adjusted mortality rate (RAMR) is used widely by healthcare agencies to evaluate hospital performance. The RAMR is insensitive to case volume and requires a confidence interval for proper interpretation, which results in a hypothesis testing framework. Unfamiliarity with hypothesis testing can lead to erroneous interpretations by the public and other stakeholders. We argue that screening, rather than hypothesis testing, is more defensible. We propose an alternative to the RAMR that is based on sound statistical methodology, easier to understand and can be used in large-scale screening with no additional data requirements.
 
Methods
We use an upper-tail probability to screen for hospitals performing poorly and a lower-tail probability to screen for hospitals performing well. Confidence intervals and hypothesis tests are not needed to compute or interpret our measures. Moreover, unlike the RAMR, our measures are sensitive to the number of cases treated.
 
Results
To demonstrate our proposed methodology, we obtained data from the New York State Department of Health for 10 Inpatient Quality Indicators (IQIs) for the years 2009-2013. We find strong agreement between the upper tail probability (UTP) and the RAMR, supporting our contention that the UTP is a viable alternative to the RAMR.
 
Conclusion
We show that our method is simpler to implement than the RAMR and, with no need for a confidence interval, it is easier to interpret. Moreover, it will be available for all hospitals and all diseases/conditions regardless of patient volume

Keywords

Main Subjects

References

  1. Centers for Medicare & Medicaid Services. Research, Statistics, Data & Systems. Historical. http://www.cms.gov. Accessed March 22, 2016. Published December 3, 2015.
  2. Center for Medicare and Medicaid Services. https://data.medicare.gov/. Accessed March 25, 2016.
  3. Your Hospital's Safety Grade. Leapfrog Hospital Safety Grade website. http://www.hospitalsafetyscore.org/your-hospitals-safety-score/. Published October 2015.
  4. Bardach N, Hibbard JH, Dudley RA. Users of public reports of hospital quality: who, what, why and how? Agency for Healthcare Research and Quality; 2011.
  5. Hibbard JH, Stockard J, Tusler M. Does publicizing hospital performance stimulate quality improvement efforts? Health Aff (Millwood). 2003;22(2):84-94.
  6. Baker DW. The effect of publically reporting hospital performance on market share and risk-adjusted mortality at high mortality hospitals. Med Care. 2003;41:729-740.
  7. Schenider EC, Lieberman T. Publicly disclosed information about the quality of healthcare: response of the public. Qual Health Care. 2001;10(2):96-103.
  8. Iezzoni LI. The risk of risk adjustment. JAMA. 1997;278:1600-1607.
  9. Thomas JW, Hofer TP. Accuracy of risk-adjusted mortality rate as a measure of hospital quality of care. Med Care. 1999;37:83-92.
  10. Dimick J, Welch HG, Timothy P. Surgical mortality as an indicator of hospital quality: the problem of small sample size. JAMA. 2004;292(7):47-51. doi:10.1001/jama.292.7.847
  11. Scott I. Where does Risk-Adjusted Mortality Fit in a Safety Measurement Program? PSNet, AHRQ website. http://psnet.ahrq.gov/perspectives/perspective/173.  Accessed June 9, 2017.
  12. Racz MJ, Sedransk J. Bayesian and frequentist methods for provider profiling using risk-adjusted assessments of medical outcomes. J Am Stat Assoc. 2010;105(489):48-58. doi:10.1198/jasa.2010.ap07175
  13. The Joint Commission website. https://www.jointcommission.org. Accessed June 9, 2017.
  14. The Common Wealth Fund: Data Brief. http://www.commonwealthfund.org/publications/data-briefs/2007/jul/measuring-hospital-performance--the-importance-of-process-measures. Accessed June 13, 2017. Published July 2007.
  15. New York State Department of Health. https://health.data.ny.gov/. Accessed October 30, 2015.
  16. Kasprak J. Connecticut General Assembly. https://www.cga.ct.gov/2006/rpt/2006-R-0741.htm.  Accessed June 11, 2017.
  17. Marang-van de Mheen PJ, Shojania KG. Simpson’s paradox: how performance measurement can fail even with perfect risk adjustment. BMJ Qual Saf. 2014;23:701-705. doi:10.1136/bmjqs-2013-002608
  18. Hoekstra R, Morey RD, Rouder JN, Wagenmakers EJ. Robust misinterpretation of confidence intervals. Psychon Bull Rev. 2014;21(5):1157-1164. doi:10.3758/s13423-013-0572-3
  19. Belia S, Fidler F, Williams J, Cumming G. Researchers misunderstand confidence intervals and standard error bars. Psychol Methods. 2005;10:389-396.
  20. Gigerenzer G. Mindless Statistics. The Journal of Socioeconomics. 2004;33:587-606.
  21. Lecoutre M-P, Poitevineau J, Lecoutre B. Even statisticians are not immune to misinterpretations of null hypothesis tests. Int J Psychol. 2003;38:37-45.
  22. Wulff HR, Andersen B, Bradenhoff P, Guttler F. What do doctors know about statistics? Stat Med. 1987;6:3-10.
  23. Scheutz F, Andersen B, Wulff HR. What do dentists know about statistics? Scand J Dent Res. 1988;96:281-7.
  24. Porter ME, Larsson S, Lee TH. Standardizing patient outcomes measurement. N Engl J Med. 2016;374(6):504-506. doi:10.1056/NEJMp1511701
  25. Bonferroni CE. Teoria statistica delle classi e calcolo delle probabilita. Pubblicazioni delle R. Instituto Superiore di Scienze Economiche e Commerciale di Fizenze ;1936.
  26. Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol. 1995;57:289-300.
  27. Hamm, RM. Selection of verbal probabilities: a solution for some problems of verbal probability expression. Organizational Behavior and Human Decision Processes. 1991;48:193-223.
  28. Hoeffding W. On the distribution of the number of successes in independent trials. Annals of Mathematical Statistics. 1956;27:713-721.
  29. Boland PJ. The Probability Distribution for the Number of Successes in Independent Trials. Communications in Statistics -Theory and Methods. 2007;36:7:1327-1331.

Files

History

  • Receive Date: 11 January 2017
  • Revise Date: 31 July 2017
  • Accept Date: 01 August 2017
  • First Publish Date: 01 April 2018

Share

How to cite

Statistics