Experiences of Using Cochrane Systematic Reviews by Local HTA Units

Introduction and Context

Due to the amount and the ongoing growth of the literature related to medical knowledge, systematic reviews (SRs) appear as a fundamental component in the process of producing relevant recommendations for clinicians and decision-makers. ^1,2 To support policy and practice decisions, Cochrane systematic reviews (CSRs) are one of the best known and most trusted sources of evidence-based in healthcare. ^3,4 Cochrane handbook described SR as a review that “attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made.” ⁴ However despite the fact that 96% of decision-makers reported that they valued SRs in the development of new guidelines and 60% consider SRs as very important when compared with other sources of evidence, it appears clearly that SRs are underutilized. ⁵ In fact, although numerous international initiatives have been taken to support the use of SRs, variation in the uptake of the latter by decision-makers persist. ⁶

We acknowledge that healthcare professionals, local policy-makers and health system managers, can face several challenges when attempting to utilize these evidences. ⁷ These include the difficulties in applying global evidence in a local clinical context by adapting evidence from SRs so that it is locally relevant, and the way in which ‘use’ can be conceptualized.

The aim of this paper is to describe how CSRs are used by local health technology assessment (HTA) units to promote efficiency in hospital decision-making. This was done by identifying facilitators and barriers to its utilisation and then by specifying what type of SRs are needed for local users. Finally, we provide elements to support a better understanding about its local applicability.

Methods

A web-based survey was conducted between August 30 and September 10, 2013 using the Surveymonkey^® platform. The survey was developed by the authors of this study and was validated by an external academic expert in survey and HTA research as regards to the relevance of the items and the univocity of the sentences. All ten local HTA units implemented in Quebec since 2001 were solicited by email (ie, the person in charge of the unit). These units were members of a local HTA units’ community of practices (CoP) of the province of Quebec, Canada. The survey was anonymous but respondents were aware that some of their answers could indicate which HTA units were interviewed. Some respondents also contacted the research team to be able to respond precisely to some questions. In addition, when the results of the survey were presented to the members of at the CoP, some of the participants spontaneously identified themselves to explain some of the results (ie, open-ended questions). To ensure accuracy, data was checked by 2 independent evaluators (TGP and CAB).

The survey included (1) Questions to describe the HTA unit (ie, date of HTA unit creation, number of members and occupation, number and type of their publications); (2) Use and knowledge of Cochrane collaboration works and services; (3) Perceptions of the respondents concerning their experience using CSRs. The ease of using and consulting the reviews, their thematic coverage, methodological characteristics and applicability in clinical settings was assessed; (4) Barriers and facilitators associated with using CSRs into HTA at the clinical level (measured with 2 open questions); (5) Satisfaction and experience of utilisation of CSRs (measured with 10 characteristics of the format of CSRs and open-ended questions). Questions about the use and satisfaction of CSRs were rated on a 4-point Likert scale (strongly agree/satisfied, somewhat agree/satisfied, somewhat disagree/unsatisfied, strongly disagree/unsatisfied). In the final section, respondents were asked to write suggestions to improve the use of CSRs in their own context.

Results

Characteristics of the Health Technology Assessment Units

The response rate was 100%. Eight of the ten HTA units were from university hospital centers and institutes and 2 other were from health and social services centers. All units were created between 2001 and 2010, including 40% in the last 2 years. Only one unit over 10 had access to a Cochrane representative in their health facility. The average team size was 6.3 [range 2-18] people (not full-time equivalent) and 69.8% were office researchers. Other members were managers or administrative staff (23.8%), librarians (3.2%), physicians or students (3.2%). HTA units had a staff of 5 people or less in 60% of case. Furthermore, most of the staff in 40% of these units was part-time. In average, the HTA units annually produced 4.7 reports [range 2-7.3] over the last 3 years, including 4.1 published reports [range 0-7.3] and 0.6 unpublished reports [range 0-3]. Half of the units published all their reports. Table 1 provides the nature of these reports.

Use and Knowledge of Cochrane Collaboration Works and Services

Almost all HTA units were familiar with CSRs (90%) (ie, one unit indicated that about half of its team was not familiar with CSRs), a large majority was aware of the existence of Cochrane meetings (80%), less known was the existence of Cochrane webinars and trainings (50%). All HTA units used CSRs in their practice, half often or always, the other half sometimes. The CSRs were used at the time of scoping and planning, and during the conduct of the HTA project (Table 2). Only one unit had participated in a Cochrane review. HTA units used CSRs in several areas, but the most common were healthcare technologies, pain, laboratory analysis, cancer, surgery and pharmacy. During the last 3 years, over 85 searches performed in the Cochrane library database for CSRs, only 21 searches were successful (24.7%). The main topics searched without success were related to pain, health economics, management, diagnostic imaging, laboratory analysis, neurology, and pharmacology.

Perception Concerning Experience Using Cochrane Systematic Reviews

The perceptions of respondents towards CSRs were generally positive (Figure 1). CSRs were deemed easily accessible through the various databases and their thematic coverage was generally considered appropriate. All respondents agreed on the importance of consulting the CSRs due the known rigorous methodology and the solid reputation of the Cochrane collaboration. The majority of respondents believed that the methods used to aggregate the various results extracted from the literature, the hierarchy of evidence found, and the evaluation of the risk of publication bias and other biases, were relevant to HTA in a healthcare setting.

However, almost all of the respondents felt that the array of study designs being included in CSRs was too narrow to meet their needs. Moreover, many respondents disagreed that CSRs contained enough data to support the decisional process and clinical practice in their setting. Also, half of the respondents disagreed with the statement that implications for practice, as described into the reviews’ conclusions, can be applied in their clinical settings (Figure 1).

Close

Figure 1. Perception of Respondents Towards Finding CSRs and Their Quality/Applicability. Abbreviations: CSRs, Cochrane Systematic Reviews; CC, Cochrane Collaboration.

Facilitators and Barriers

Facilitators

The respondents identified 4 facilitators for using CSRs. The most often reported was the rigor of the methods used by the Cochrane collaboration. This methodology was deemed “rigorous,” “proven and constant,” “complete,” and “systematic.” This was followed by the clarity in the presentation of CSRs. Respondents also considered that the data presented in CSRs were “highly reliable” and the data analysis of “high quality.” Finally, CSRs were easy to find in bibliographical databases and main web-search engines (Google, Yahoo, Bing, etc).

Barriers

Four barriers were identified by respondents. The first was related to the fact that CSRs tend to favor the data extracted from randomized clinical trials (RCTs), while often, the issues being analysed by the HTA units involve emerging technologies and interventions, on which none or few RCTs have been published. In this setting, one respondent noted that “Cochrane reviews are excellent. The problem is that the topics we face do not easily lend themselves to SRs utilising the Cochrane method because of a lack of evidence.” A second barrier was that some respondents did not find CSRs relevant to the themes they assess. The third barrier stated was that the data presented in CSRs are withdrawn from the context into which they were observed. One respondent noted that “the analysis gives more weight to statistical analysis than to clinical reality (eg, in the choice of indicators).” In addition, CSRs collected data from emerging countries whose contexts differ enormously from that of Quebec. Finally, respondents considered that CSRs should include studies of a larger array of methodological designs, like observational and qualitative studies, as well as grey literature. The inclusion of data from such studies would contribute to deepen the analysis pertaining to safety and clinical effectiveness and assess the implementation context of technologies and intervention modes.

Satisfaction and Experience Using the CSRs in HTA in a Clinical Setting

The analysis of the data revealed that respondents were generally satisfied with the format of CSRs (Figure 2). The open-ended question revealed that the use of CSRs by members of their unit was an enriching, satisfying, useful experience, and that it reduced research tasks in some cases. They were enthusiastic when they found CSRs relevant to their topic, but disappointed when it was not applicable. However, some respondents were dissatisfied with the description of the context into which the interventions assessments were conducted and how the reviews themselves were carried out.

Close

Figure 2. Satisfaction and Experience Using the CSR in HTA in a Clinical Setting. Abbreviations: CSR, Cochrane systematic review; HTA, health technology assessment.

Almost all HTA units considered that CSRs can be incorporated into work in local HTA (90%), but a large majority considered that CSRs did not prevent their unit from conducting their own SR (90%). Some specified that research needed to be expanded to include other data, that in some cases the level of evidence was insufficient and some outcomes were missing. Some units systematically use the CSRs for each evaluation project, coupled with other articles with less controlled studies. Others were especially interested in quick reviews, mapping, and scoping ones.

Discussion

This survey showed that CSRs have a very good reputation and were consulted by HTA unit members. However, only half of the units considered that CSRs contained enough information to help in the local decision-making process and that their conclusions may be relevant to their own clinical settings.

Dobbins et al reported that a majority of decision-makers used SRs in the development of new guidelines and consider SR as very important when compared with other forms of evidence. ^8,9 Actually, our results showed that, in Quebec, all HTA units consulted CSRs and considered that CSRs are useful and should be incorporated in their work. Two units even mentioned that the existing CSRs have avoided performing a new SR. On the other hand, a large majority considered that they still need to conduct a SR including non-RCTs, by mentioning that the level of evidence in CSRs was insufficient, with too few SR in the mental health, physical readaptation and rehabilitation area, and that they should expand the inclusion criteria to perform reviews more in line with the real world.

In this survey, the main facilitators that were reported for the use of CSRs were their methodological rigor, a clear and standardized presentation, a high reliability, and quality of analyzes. This echoes to the fact that CSRs are actually recognized in the literature for their rigorous methodology and concise summaries. ^3,10-13 Another facilitator is that CSRs were easy to find in bibliographical databases and by using main web-search engines (Google, Yahoo, Bing, etc). Wallace et al published a SR of facilitators to improve utilisation of SRs and meta-analyses. ⁷ There are many facilitators, but the most common are that SR improves knowledge; a content that includes benefits, harms, and cost; training in use and peer-group support. For Dobbins et al, managers or directors use SRs more than medical and associate medical officers of health. Other predictor of use SRs were: expecting to use a SR in the future; that they overcame the barrier of limited critical appraisal skills; and perceived the reviews as being easy to use. ⁹

Although SRs and CSRs are well-recognized, they are still underused. ⁵ The main barriers we found in this survey were that the topics performed by HTA units often do not lend themselves well to a SR, a lack of evidence in CSRs, and that the existing CSRs were not always relevant to the themes they assess. Moreover, the data are often out of local context and do not sufficiently consider the clinical reality. More generally, CSRs assess the level of evidence associated to a causal relationship but do not consider the socio-cultural and organisational context in which health and social services interventions are conducted. This is reinforced by the fact that CSRs generally draw their evidence from RCTs. As a matter of fact, the appraisal of a larger array of methodological designs, like observational and qualitative studies, as well as grey literature, would help to deepen the analysis of safety and clinical effectiveness and assess the implementation context of technologies and intervention modes. However, these designs are deemed of lower methodological quality by the Cochrane collaboration while they are generally accepted in HTA standards. ^14-18 Moreover, the inclusion of qualitative studies would help strengthen the appraisal of the implementation of technologies and intervention modes as well as its context. ¹⁹ Lavis et al ¹¹ also made this remark that SRs on other types of studies like qualitative or social services studies are less frequent. ^20-23 Later he reported that policy-makers and stakeholders need many types of SRs, reviews of observational studies, of qualitative studies, of effectiveness studies, reviews of economic evaluations, etc. and that the number of these reviews is increasing. ¹² All this echoes the debate surrounding the paradigm approach chosen by the Cochrane collaboration in favor of experimental designs and particularly RCTs. ²⁴ Considering that decision-makers cannot simply make a choice based on experimental data, but also need data about the context, experiential knowledge, values and preferences of users and professionals, ² this may explain why CSRs have a limited impact on clinical practice and the organization and delivery of healthcare services. ²⁵

Limitations of this survey include the limited number of respondents and the generalizability of the results outside Quebec. However, at the time of the survey, there were only ten HTA units implemented in Quebec’s hospitals and social services centres. As all respondents answered the survey, this limit became a strength by providing the opinion of the entire population considered (ie, internal validity). The limited amount of data allowed us to do descriptive analyzes and if these results cannot be generalized outside the province of Quebec, they provide a good idea of the use of CSRs by local HTA units. Another limitation is that the survey was conducted 6 years ago. At the time of the investigation, the CoP was essentially composed of HTA units in physical health, but since, new social service units were created. As there are very little CSRs in this area, probably these units would have been even more disappointed. Medical science produces the majority of the literature for CSRs, but in the social sciences, the use of meta-analysis is rapidly increasing. ²⁶ If the Cochrane collaboration is primarily associated with healthcare studies, its international equivalent for social care (as well as education, crime and justice) is the Campbell Collaboration. ²⁷ However, despite an increase in SRs in social care in the last decade, the number remains limited. ^27,28

As a result, a few suggestions to improve the use of CSRs by the HTA units would be: (1) To include non RCT; (2) To consider additional factors in the appraisal (eg, clinical heterogeneity, economics, organizational impacts, ethics); (3) To extend topics on social services and emerging technologies; (4) To develop the understanding about the applicability of CSRs to inform decision-making. If not, the future of CSRs in the HTA environment could be to promote best practices by summarizing the evidence only, or to be used as other sources of information in HTA report. ²⁹ However, since this survey was conducted, there have been several echoes from the Cochrane collaboration saying that they would make an effort to include other types of studies than randomised control trial. ³⁰ That’s a good start, even if the number of CSRs using qualitative and mixed methods studies is still very low and that contextualisation and transferability of evidence will remain an important challenge. ³¹

Conclusion

To our knowledge, this is the first study that evaluated the use of CSRs by local HTA units to promote efficiency in health decision-making. The results indicate that HTA units use and consult CSRs in their work, but that they generally complement it with other articles in the literature or produce an original SR according to their specific objectives. In some cases these units even produced primary data and collaborated with researchers to cope with the lack of evidence in the scientific literature. ^32-35 Many respondents supported that CSRs contained not enough data to support the decisional process and clinical practice. As a future avenue, it would be interesting to see if the opinion of the HTA units about the use of the CSRs has changed over the last 6 years.

Acknowledgements

We would like to thank all members of the HTA community of practice in Quebec for their participation in the study, as well as Julie Dussault, Alison Sinclair, Marie-Pascale Pomey, and Nathalie Carrier for their valuable comments. TGP is member of the FRQS-funded Centre de recherche de l’IUSMM. TGP is fellow of the FRQS.

Ethical issues

In accordance with the local ethics committee of participating organizations and the main policies governing research ethics in Health and Social Services in Québec, Canada, it was deemed not necessary to obtain an ethics approval for the project since: (1) It was proposed by the CoP and unanimously accepted by all its members; (2) It was not about personal data but about the professional use of CSRs by different HTA units. Under these circumstances, it was considered that no risks were associated to this project.

Competing interests

Authors declare that they have no competing interests.

Authors’ contributions

All authors contributed to conception and design, analysis and interpretation of data, and drafting and critical revision of the manuscript. TGP, MR, CAB, CS, and SSJ contributed to acquisition of data. TGP, CAB, and SD contributed to statistical analysis. CS contributed to administrative and technical support. TGP, MR, CAB, and PD contributed to supervision of the study.

Authors’ affiliations

¹Department of Management, Evaluation and Health Policy, School of Public Health, University of Montreal, Montréal, QC, Canada. ²Centre de Recherche de l’Institut Universitaire en Santé Mentale de Montréal, CIUSSS de l’Est de l’Île de Montréal, Montréal, QC, Canada. ³HTA Unit, CHU de Québec – Université Laval, Québec, QC, Canada. ⁴Department of Multidisciplinary Services, Clinical Quality Division, CIUSSS de l’Estrie-CHUS, Sherbrooke, QC, Canada. ⁵HTA Unit, CHUM, Montréal, QC, Canada. ⁶HTA Unit, CIUSSS de la Capitale-Nationale, Québec, QC, Canada. ⁷HTA Unit, CIUSSS de l’Estrie – CHUS, Sherbrooke, QC, Canada. ⁸Department of Medicine, Faculty of Medicine and Health Science, University of Sherbrooke, Sherbrooke, QC, Canada.

References

1. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ 1996; 312(7023):71-72. doi: 10.1136/bmj.312.7023.71 [Crossref] [ Google Scholar]
2. Poder TG, Bellemare CA. Importance of contextual data in producing health technology assessment recommendations: a case study. Int J Technol Assess Health Care 2018; 34(1):63-67. doi: 10.1017/s0266462317004469 [Crossref] [ Google Scholar]
3. About Cochrane Reviews. Cochrane Library website. https://www.cochranelibrary.com/about/about-cochrane-reviews. Accessed November 15, 2018.
4. Green S, Higgins J, Alderson P, Clarke M, Mulrow C, Oxman A. Chapter 1: Introduction. In: Higgins J, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions. West Sussex, England: The Cochrane Collaboration and John Wiley; 2008.
5. Tricco AC, Tetzlaff J, Moher D. Knowledge synthesis. In: Straus SE, Tetroe J, Graham ID, eds. Knowledge Translation in Health Care. Wiley-Blackwell; 2013:29-49. doi:10.1002/9781118413555.ch04
6. Marquez C, Johnson AM, Jassemi S. Enhancing the uptake of systematic reviews of effects: what is the best format for health care managers and policy-makers? a mixed-methods study. Implement Sci 2018; 13(1):84. doi: 10.1186/s13012-018-0779-9 [Crossref] [ Google Scholar]
7. Wallace J, Byrne C, Clarke M. Making evidence more wanted: a systematic review of facilitators to enhance the uptake of evidence from systematic reviews and meta-analyses. Int J Evid Based Healthc 2012; 10(4):338-346. doi: 10.1111/j.1744-1609.2012.00288.x [Crossref] [ Google Scholar]
8. Dobbins M, DeCorby K, Twiddy T. A knowledge transfer strategy for public health decision makers. Worldviews Evid Based Nurs 2004; 1(2):120-128. doi: 10.1111/j.1741-6787.2004.t01-1-04009.x [Crossref] [ Google Scholar]
9. Dobbins M, Cockerill R, Barnsley J. Factors affecting the utilization of systematic reviews A study of public health decision makers. Int J Technol Assess Health Care 2001; 17(2):203-214. doi: 10.1017/s0266462300105069 [Crossref] [ Google Scholar]
10. Perrier L, Mrklas K, Lavis JN, Straus SE. Interventions encouraging the use of systematic reviews by health policymakers and managers: a systematic review. Implement Sci 2011; 6:43. doi: 10.1186/1748-5908-6-43 [Crossref] [ Google Scholar]
11. Lavis J, Davies H, Oxman A, Denis JL, Golden-Biddle K, Ferlie E. Towards systematic reviews that inform health care management and policy-making. J Health Serv Res Policy 2005; 10 Suppl 1:35-48. doi: 10.1258/1355819054308549 [Crossref] [ Google Scholar]
12. Lavis JN. How can we support the use of systematic reviews in policymaking?. PLoS Med 2009; 6(11):e1000141. doi: 10.1371/journal.pmed.1000141 [Crossref] [ Google Scholar]
13. Alper BS, Hand JA, Elliott SG. How much effort is needed to keep up with the literature relevant for primary care?. J Med Libr Assoc 2004; 92(4):429-437. [ Google Scholar]
14. Velasco M, Perleth M, Drummond M. Best practice in undertaking and reporting health technology assessments Working group 4 report. Int J Technol Assess Health Care 2002; 18(2):361-422. doi: 10.1017/s0266462302000284 [Crossref] [ Google Scholar]
15. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009; 6(7):e1000097. doi: 10.1371/journal.pmed.1000097 [Crossref] [ Google Scholar]
16. Kung J, Chiappelli F, Cajulis OO. From systematic reviews to clinical recommendations for evidence-based health care: validation of revised assessment of multiple systematic reviews (R-AMSTAR) for grading of clinical relevance. Open Dent J 2010; 4:84-91. doi: 10.2174/1874210601004020084 [Crossref] [ Google Scholar]
17. Whiting P, Savović J, Higgins JP. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 2016; 69:225-234. doi: 10.1016/j.jclinepi.2015.06.005 [Crossref] [ Google Scholar]
18. Higgins JP, Altman DG, Gøtzsche PC. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011; 343:d5928. doi: 10.1136/bmj.d5928 [Crossref] [ Google Scholar]
19. Kringos DS, Sunol R, Wagner C. The influence of context on the effectiveness of hospital quality improvement strategies: a review of systematic reviews. BMC Health Serv Res 2015; 15:277. doi: 10.1186/s12913-015-0906-0 [Crossref] [ Google Scholar]
20. Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy 2005; 10(1):45-53. doi: 10.1177/135581960501000110 [Crossref] [ Google Scholar]
21. Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy 2005; 10 Suppl 1:6-20. doi: 10.1258/1355819054308576 [Crossref] [ Google Scholar]
22. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004; 82(4):581-629. doi: 10.1111/j.0887-378X.2004.00325.x [Crossref] [ Google Scholar]
23. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review--a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy 2005; 10 Suppl 1:21-34. doi: 10.1258/1355819054308530 [Crossref] [ Google Scholar]
24. Kongsted HC, Konnerup M. Are more observational studies being included in Cochrane Reviews?. BMC Res Notes 2012; 5:570. doi: 10.1186/1756-0500-5-570 [Crossref] [ Google Scholar]
25. Bunn F, Trivedi D, Alderson P, Hamilton L, Martin A, Iliffe S. The impact of Cochrane Systematic Reviews: a mixed method evaluation of outputs from Cochrane Review Groups supported by the UK National Institute for Health Research. Syst Rev 2014; 3:125. doi: 10.1186/2046-4053-3-125 [Crossref] [ Google Scholar]
26. Davis J, Mengersen K, Bennett S, Mazerolle L. Viewing systematic reviews and meta-analysis in social research through different lenses. Springerplus 2014; 3:511. doi: 10.1186/2193-1801-3-511 [Crossref] [ Google Scholar]
27. Kelly E. Systematic and just: the use of a systematic review methodology in social work research. Soc Work Soc Sci Rev 2015; 15(3):72-85. [ Google Scholar]
28. Rutter D, Francis J, Coren E, Fisher M. SCIE Systematic Research Reviews: Guidelines. 2nd ed. Social Care Institute for Excellence (SCIE); 2011.
29. Pedersen VH, Dagenais P, Lehoux P. Multi-source synthesis of data to inform health policy. Int J Technol Assess Health Care 2011; 27(3):238-246. doi: 10.1017/s0266462311000213 [Crossref] [ Google Scholar]
30. Anglemyer A, Horvath HT, Bero L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev 2014(4):MR000034. doi: 10.1002/14651858.MR000034.pub2 [Crossref]
31. Munthe-Kaas H, Nøkleby H, Nguyen L. Systematic mapping of checklists for assessing transferability. Syst Rev 2019; 8(1):22. doi: 10.1186/s13643-018-0893-4 [Crossref] [ Google Scholar]
32. Poder TG. Using the health technology assessment toolbox to facilitate procurement: the case of smart pumps in a Canadian hospital. Int J Technol Assess Health Care 2017; 33(1):54-62. doi: 10.1017/s0266462317000125 [Crossref] [ Google Scholar]
33. Poder TG, Godbout ST, Bellemare C. Dual vs single computer monitor in a Canadian hospital Archiving Department: a study of efficiency and satisfaction. Health Inf Manag 2011; 40(3):20-25. doi: 10.1177/183335831104000303 [Crossref] [ Google Scholar]
34. Poder TG, Pruneau D, Dorval J. Pressure infusion cuff and blood warmer during massive transfusion: an experimental study about hemolysis and hypothermia. PLoS One 2016; 11(10):e0163429. doi: 10.1371/journal.pone.0163429 [Crossref] [ Google Scholar]
35. Chen Z, Deblois S, Toporowicz K. Yield of CT pulmonary angiography in the diagnosis of acute pulmonary embolism: short report. BMC Res Notes 2019; 12(1):41. doi: 10.1186/s13104-019-4076-8 [Crossref] [ Google Scholar]