Some Notes on Critical Appraisal of Prevalence Studies; Comment on: “The Development of a Critical Appraisal Tool for Use in Systematic Reviews Addressing Questions of Prevalence”

Document Type : Commentary

Author

Immunization Unit, Robert Koch Institute, Berlin, Germany

Abstract

Decisions in healthcare should be based on information obtained according to the principles of Evidence-Based Medicine (EBM). An increasing number of systematic reviews are published which summarize the results of prevalence studies. Interpretation of the results of these reviews should be accompanied by an appraisal of the methodological quality of the included data and studies. The critical appraisal tool for prevalence studies developed and tested by Munn et al. comprises 10 items and aims at targeting all kinds of prevalence studies. This comment discusses the pros and cons of different designs of quality appraisal tools and highlights their importance for systematic reviews of prevalence studies. Beyond piloting, which has been performed in the study by Munn et al., it is suggested here that the validity of the tool should be tested, including reproducibility and inter-rater reliability. It is concluded that studies as the one by Munn et al. will help to establish a critical understanding of methodological quality and will support the use of systematic reviews of non-intervention studies for health policy making.

Keywords

Main Subjects

References

  1.  

    1. Evidence-Based Medicine Working G. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA 1992; 268: 2420-5. 
    2. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ 1996; 312: 71-2. 
    3. Barendregt JJ, Doi SA, Lee YY, Norman RE, Vos T. Meta-analysis of prevalence. J Epidemiol Community Health 2013; 67: 974-8. doi: 10.1136/jech-2013-203104
    4. Spencer FA, Iorio A, You J, Murad MH, Schunemann HJ, Vandvik PO, et al. Uncertainties in baseline risk estimates and confidence in treatment effects. BMJ 2012; 345: e7401. doi: 10.1136/bmj.e7401
    5. Kelly M, Morgan A, Ellis S, Younger T, Huntley J, Swann C. Evidence based public health: A review of the experience of the National Institute of Health and Clinical Excellence (NICE) of developing public health guidance in England. Soc Sci Med 2010; 71: 1056-62.  doi: 10.1016/j.socscimed.2010.06.032
    6. Katrak P, Bialocerkowski AE, Massy-Westropp N, Kumar S, Grimmer KA. A systematic review of the content of critical appraisal tools. BMC Med Res Methodol 2004; 4: 22. doi: 10.1186/1471-2288-4-22
    7. Shamliyan T, Kane RL, Dickinson S. A systematic review of tools used to assess the quality of observational studies that examine incidence or prevalence and risk factors for diseases. J Clin Epidemiol 2010; 63: 1061-70.  doi: 10.1016/j.jclinepi.2010.04.014
    8. Harder T, Takla A, Rehfuess E, Sanchez-Vivar A, Matysiak-Klose D, Eckmanns T, et al. Evidence-based decision-making in infectious diseases epidemiology, prevention and control: matching research questions to study designs and quality appraisal tools. BMC Med Res Methodol 2014; 14: 69. doi: 10.1186/1471-2288-14-69
    9. Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol 2007; 36: 666-76. doi: 10.1093/ije/dym018
    10. Munn Z, Moola S, Riitano D, Lisy K. The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence. Int J Health Policy Manag 2014; 3: 123-8.  doi: 10.15171/ijhpm.2014.71
    11. Hoy D, Brooks P, Woolf A, Blyth F, March L, Bain C, et al. Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement. J Clin Epidemiol 2012; 65: 934-9. doi: 10.1016/j.jclinepi.2011.11.014
    12. Loney PL, Chambers LW, Bennett KJ, Roberts JG, Stratford PW. Critical appraisal of the health research literature: prevalence or incidence of a health problem. Chronic Dis Can 1998; 19: 170-6. 
    13. Health NCCfE. A primer for evaluating the quality of studies on environmental health critical appraisal of cross-sectional studies. 2014. Available from: http://www.ncceh.ca/sites/default/files/Critical_Appraisal_Cross_Sectional_Studies_Aug_2011.pdf
    14. Huwiler-Muntener K, Juni P, Junker C, Egger M. Quality of reporting of randomized trials as a measure of methodologic quality. JAMA 2002; 287: 2801-4. 
    15. Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011; 343: d5928. doi: 10.1136/bmj.d5928
    16. Hartling L, Milne A, Hamm MP, Vandermeer B, Ansari M, Tsertsvadze A, et al. Testing the Newcastle Ottawa Scale showed low reliability between individual reviewers. J Clin Epidemiol 2013; 66: 982-93. doi: 10.1016/j.jclinepi.2013.03.003
    17. Cho MK, Bero LA. Instruments for assessing the quality of drug studies published in the medical literature. JAMA 1994; 272: 101-4. 
    18. Hartling L, Hamm MP, Milne A, Vandermeer B, Santaguida PL, Ansari M, et al. Testing the risk of bias tool showed low reliability between individual reviewers and across consensus assessments of reviewer pairs. J Clin Epidemiol 2013; 66: 973-81. doi: 10.1016/j.jclinepi.2012.07.005

Files

History

  • Receive Date: 01 October 2014
  • Revise Date: 11 October 2014
  • Accept Date: 10 October 2014
  • First Publish Date: 10 October 2014

Share

How to cite

Statistics