Clinical Decision Support and New Regulatory Frameworks for Medical Devices: Are We Ready for It? - A Viewpoint Paper

Use of Software as a Medical Device (SaMD) is continuously increasing. SaMD is defined by the International Medical Device Regulators Forum (IMDRF) as software intended to be used for medical purposes without being part of a hardware medical device.¹ SaMD is software that can perform complex clinical tasks such as diagnosing medical conditions, suggesting treatments and informing clinical management.¹

In light of the rapidly changing digital health landscape, policy makers have acknowledged the need for an updated regulatory framework to ensure and promote safe innovation. In the United States, such a regulatory response was given in the 21st Century Cures Act, which was enacted in December 2016.² In the European Union (EU), the Medical Device Regulation (MDR) followed in May 2017.³ This Regulation was due to be enforceable in all member states as of May 2020, but this was postponed with a year because of the coronavirus disease 2019 (COVID-19) pandemic.⁴ Both laws aim to set high quality and safety standards for medical devices and improved transparency and traceability.^2,3

In this paper, we will focus on the impact of these legislations on clinical decision support (CDS) software as a specific type of SaMD. We define CDS as any software system that integrates personal patient data with external sources of medical knowledge to assist healthcare professionals in their decision-making process.⁵ Such an external source may be well-established information such as clinical guidelines, but it can also be information inferred by an algorithm using machine learning or artificial intelligence techniques. CDS can be either stand-alone or integrated in another medical device.⁵

21st Century Cures Act: Device CDS and Non-device CDS

In the United States, controversy was raised around the implications of the Cures Act on CDS.^6,7 The first major problem was that it was initially not clear which types of CDS would be regulated under the Cures Act and which ones would be exempted. Secondly, the new requirements were not always proportionate to the risk a particular CDS function might pose.

The Food and Drug Administration (FDA) responded with the publication of a guidance on CDS software on September 27, 2019.⁸ In this document the FDA clarified their position on regulating different kinds of CDS software functions using a risk-based approach as suggested by the IMDRF Framework.^8,9 In the guidance document, CDS is categorized into ‘Device CDS’ and ‘Non-device CDS’. Non-Device CDS is CDS that is exempted from the definition of medical device because it meets all of the following four criteria⁸:

1. Not intended to acquire, process, or analyze a medical image or a signal from an in vitro diagnostic device or a pattern or signal from a signal acquisition system;

2. Intended for the purpose of displaying, analyzing, or printing medical information about a patient or other medical information (such as peer-reviewed clinical studies and clinical practice guidelines);

3. Intended for the purpose of supporting or providing recommendations to a healthcare professional about prevention, diagnosis, or treatment of a disease or condition; and

4. Intended for the purpose of enabling such healthcare professional to independently review the basis for such recommendations that such software presents so that it is not the intent that such healthcare professional rely primarily on any of such recommendations to make a clinical diagnosis or treatment decision regarding an individual patient.

An example of a Non-Device CDS that the FDA considers to be exempted from regulation is software that identifies drug-drug interactions based on the recommendation of reliable medical sources.⁸ CDS for drug-drug interaction screening is often knowledge-based, comparing information from one or more validated drug-drug interaction databases to the medication a patient is currently taking and newly prescribed medication.

Unfortunately, current CDS for drug-drug interaction screening is often too sensitive leading to a high alert burden with alert fatigue and high override rates.^10,11 Improving the specificity of CDS for drug-drug interaction screening is an essential part of ongoing efforts to improve CDS’s effectiveness.^10,12 Algorithms may rely on traditional statistical regression models or on more advanced machine learning or artificial intelligence techniques. For both, it is not clear whether the healthcare professional is truly capable of independently reviewing the basis of the recommendation, the key criterion that decides whether the CDS function is considered Device CDS or Non-device CDS. The FDA states that the manufacturer should describe the algorithm’s underlying data and include plain language descriptions of the logic or rationale used to produce a certain recommendation. When complex algorithms are used to determine the risk level of a drug-drug interaction, this would mean that the specific elements used (eg, lab values and comorbidities), should be displayed on the alert screen. When the logic of the algorithm is also explained and available to the healthcare professional (directly or on demand), the fourth criterion is fulfilled, and thus the software is exempted from the Cures Act. As Evans and Ossorio pointed out, the software manufacturer only needs to intend and does not need to succeed for its software to be transparent and explainable in its recommendations.¹³ It seems that the fourth criterion should thus be interpreted as whether the healthcare practitioner can theoretically understand or evaluate the basis for a certain recommendation.

The FDA still has to further clarify how they will assess whether a certain CDS software is explained or not. This highly influences whether the manufacturer has to comply with FDA regulation or is exempted. In case the FDA would only exempt the simplest CDS software, the opposite effect of the intention of the Cures Act may be achieved, namely significant delayed access to new innovations and advances for patients.

21st Century Cures Act: Risk Classification

Acknowledging the balance between timely patient access and safety, FDA adopted the risk-based classification of SaMD as proposed by IMDRF.^8,9 This classification is based on two main factors: (a) the significance of the information provided by a SaMD to a certain type of healthcare decision, and (b) the state of the patient’s healthcare situation or condition, resulting in four risk categories. Category IV is considered the highest risk category, whereas SaMD with the lowest risk are classified as category I (Table).

The guidance specifies that SaMD functions that drive clinical management or that treat or diagnose are not considered CDS as defined in the Cures Act, because criterion (3) is not met. Hence, CDS functions always belong to category I or II (Figure, section A). CDS functions that inform clinical management for non-serious conditions with patients or caregivers as intended users who can independently review the basis of the recommendations, are considered to be class I and low risk CDS functions. An example of this type of CDS is software that provides patients or caregivers a prioritized list of over-the-counter medications matching with their notified symptoms for a non-serious condition like common cold. Likewise, CDS functions for a non-serious condition intended for healthcare professionals that do not meet criterion (4), because the recommendations are not designed to be independently reviewed, are considered low risk. An example of such a CDS function would be an algorithm that uses patient-specific data such as blood test results and medication information to alert healthcare practitioners of cholesterol management issues. For this type of Device CDS functions, FDA does not intend to enforce compliance with the Cures Act (Figure, section A). Instead, FDA will focus its regulatory oversight on higher risk Device CDS functions informing clinical management for serious and critical healthcare conditions, like for example an unexplained algorithm that identifies hospitalized patients at increased risk of postoperative cardiovascular events.

Close

Figure. Infographic on Food and Drug Administration Policy on Clinical Decision Support According to the Food and Drug Administration Guidance Specifically for Clinical Decision Support8 (A) and on the Classification of Medical Device Software According to the European Union Medical Device Regulation (B). Abbreviations: CDS, clinical decision support; FDA, Food and Drug Administration; HCP, healthcare professional; MDR, Medical Device Regulation; EU, European Union. “FDA Enforcement Discretion” indicates that the FDA does not intend to enforce compliance, and “FDA oversight focus” indicates that the focus is on regulating CDS software by the FDA. In the United States, CDS functions belong to category I or II depending on the health condition and type of intended user. In the EU, medical device software with the lowest risk is classified as class I device, whereas class III represents the highest risk category. Only class I devices can be self-certified, other devices must be certified by a notified body.

Europe’s Medical Device Regulation

In Europe a similar medical device law, the MDR, was implemented in May 2021. Instead of the term SaMD, the term medical device software is used. According to the definition of medical devices any software with the purpose of: (1) diagnosis, prevention, monitoring, prediction, prognosis, treatment or alleviation of disease; (2) diagnosis, monitoring, treatment, alleviation of, or compensation for, an injury or disability; (3) investigation, replacement or modification of the anatomy or of a physiological or pathological process or state; and (4) providing information by means of in vitro examination of specimens derived from the human body, including organ, blood and tissue donations, falls under the new legislation.³ Under the MDR, software vendors should certify their products via notified bodies.^3,14 These notified bodies assess the conformity of the software before being placed on the market by evaluating manufacturers’ technical documentation and quality management system with an increasing level of scrutiny based on the devices’ classification.^3,15 In annex VIII of the MDR, rule 11 states that software intended to provide information that is used to take decisions with diagnostic or therapeutic purposes is classified as Class IIa, except if such decisions may cause a serious deterioration of a person’s health state or a surgical intervention (Class IIb) or when they can result in death or an irreversible deterioration of a person’s state of health (class III) (Figure, section B). Software that does not lead to decisions on diagnosis or therapy and that does not monitor physiological processes can be classified as a Class I device, allowing self-certification. An example of a class I software is an application calculating the user’s fertility status to predict ovulation. Moreover, the MDR takes on a product lifecycle-focused approach resulting in post-certification monitoring, annual surveillance audits of the manufacturers by notified bodies and re-certification procedures at least every five years.

FDA vs MDR for CDS Functionalities

Like the FDA, the MDR has leveraged the IMDRF risk-based framework for medical device software classification, but the MDR is more stringent: all software functionalities that classify as class I devices under FDA regulation are classified as at least class IIa devices under MDR regulation. The previous example of CDS for drug-drug interaction screening that is exempted from the Cures Act is at least classified as class IIa medical software in EU’s MDR. Since the application is used to guide the treatment of the patient, it will either belong to class IIa, IIb or III depending on how developers describe and notified bodies evaluate the intended use. Not detecting a drug-drug interaction may have no consequences but can also result in adverse drug events and worst case even in death. The highest potential risk will determine the risk profile. However, the risk-based categorization seems not fully defined, thereby creating opportunity for subjective interpretation.

Like the US case, different European organizations raised awareness around the idea of regulating CDS applications too strongly.^16,17 The FDA acknowledged such concerns with the draft guidance on CDS, but so far, the EU has not yet issued a specific statement concerning MDR and CDS risk classification. Most organizations think MDR will hamper software development, will limit their focus of producing software because of the administrative burden and will limit the access of these CDS applications to the market.^16,17 The authors of this paper agree that it is a good idea to regulate medical software, but one should be aware of introducing a possible inequity between “large” and “small” companies and/or academic institutions that develop medical software in fulfilling the administrative part in terms of quality and safety. Both human resources and financial constraints may for example prevent hospitals from developing a homegrown healthcare information system with integrated medical device software functionalities (eg, CDS).

The IMDRF is a consultative body between international stakeholders aiming to accelerate international medical device regulatory harmonization and convergence. IMDRF has the ability to play a leading role in the harmonization process of the USA’s and Europe’s viewpoint on CDS regulation. Countries and continents can aim to adopt similar definitions, but the implementation of regulatory frameworks remains the responsibility of the FDA for the Cures Act and EU government for the MDR legislation.

Remaining Issues

In our opinion neither American nor European authorities have clarified all issues concerning the regulation of CDS. The United States has gone through a process of revising the Cures Act, where over time risk categories were included and lower risk categories were assigned to medical software for which a user can understand or evaluate the basis of a clinical recommendation.

In Europe, the MDR has not undergone similar major changes (yet). It is believed that EU’s regulation is larger in scope compared to the United States.¹⁸ Returning to the example of CDS for drug-drug interactions, the authors of this paper believe that the exemption in the United States and stringent regulation in Europe can lead to less CDS innovation in Europe compared to the United States. Currently, only 23 notified bodies spread over 11 different countries in the EU are accredited to regulate all applications for medical devices.¹⁹ Moreover, these notified bodies are responsible for evaluating both software applications and all other medical devices. Software vendors in countries without a notified body, are forced to use notified bodies located in another country and may not be able to apply in their own language. This also implies that the number of applications will be larger for these notified bodies, leading to a larger workload and potentially delayed market access to new technologies.

When looking at the risk classification both in the United States and Europe, it is not clear how this classification is objectively assessed. In our opinion, medical software manufacturers will seek to certify their application in the lowest risk class possible. As mentioned earlier, this classification depends on the documentation describing the intended use and the risk evaluation procedure and on the evaluation of this documentation by the notified body.

Furthermore, CDS may also include black-box solutions incorporating artificial intelligence. Do these solutions then automatically fall in the highest risk category, since the basis of that clinical decision is not directly understandable, or will the risk categorization depend more on the type of application where the artificial intelligence is used for? Or will the display of all data input elements be considered sufficient to independently review the basis of a recommendation? Will the MDR also take this approach? Uncertainty also remains on the threshold for recertification. Does an update of the algorithms require a recertification by a notified body, even if the intended use remains the same?

These are questions that require answers before being able to apply for certification. A guidance document with an extensive list of applications and the corresponding risk category could make it easier and more comprehensible for (future) developers to understand the risk categorization. In the United States, the FDA already created such a list in which manufacturers and users of these devices can find to what risk class the device belongs according to the 21st Century Cures Act.²⁰ However, the authors of this paper note that two medical software devices with the same intended use may differ substantially leading to different risk evaluations. For example, software for checking drug-drug interactions might be knowledge-based or non-knowledge based. The former category uses fixed IF-THEN rules that are programmed in the system to decide on an action (eg, alert), while the decisions of the latter category are based on more complex algorithms using for example, a statistical model (eg, logistic regression model) or a black-box machine learning algorithm where the basis of the decision is much more difficult to track. Both solutions have the same purpose but may have to evolve in a different risk evaluation in our opinion.

Designing a regulatory framework that achieves the right balance between promoting innovation and fast market access on one side and ensuring safety and quality on the other side is very challenging. Both the United States and Europe responded to the initial need for a new regulatory framework. Future direction should go to providing sufficient guidance on how to fulfill a certification procedure from start to finish. Many consultancy companies are taking advantage of the complexity by offering assistance, but this places large costs on small market players. The goal of promoting innovation and fast market access might be bypassed. The authors understand that different risk evaluations will lead to different procedures, but manufacturers should at least find the appropriate information on where to start and what exactly needs to be done in order to meet the legislation requirements. Within the next few years, we will be able to evaluate the effect of these laws on the clinical translation of innovative CDS software systems.

Ethical issues

Not applicable.

Competing interests

Authors declare that they have no competing interests.

Authors’ contributions

All authors contributed to the theoretical concept of this viewpoint paper. SVL and KMM drafted the first version of the manuscript and all three authors critically reviewed the manuscript and gave their approval before submission.

Funding

KMM is supported by a grant of the Research Foundation Flanders (FWO) under grant number [1S39820N]. This funding body had no role in any part of the work.

References

1. IMDRF SaMD Working Group. Software as a Medical Device (SaMD): Key Definitions. http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-131209-samd-key-definitions-140901.pdf. Accessed March 22, 2021. Published 2013.
2. U.S. Government Publishing Office. Public Law 114-255 - 114th Congress (in short: 21st Century Cures Act). https://www.govinfo.gov/app/details/PLAW-114publ255. Accessed March 22, 2021. Published 2016.
3. European Parliament and Council. Regulation (EU) 2017/745 (in short: Medical Device Regulation (MDR)). https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:02017R0745-201705052017. Accessed March 22, 2021. Published 2017.
4. European Union. Regulation (EU) 2020/561 of the European Parliament and of the Council of 23 April 2020 amending Regulation (EU) 2017/745 on medical devices, as regards the dates of application of certain of its provisions. http://data.europa.eu/eli/reg/2020/561/oj. Accessed March 22, 2021. Published 2020.
5. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3:17. doi: 10.1038/s41746-020-0221-y [Crossref] [ Google Scholar]
6. Kracov DA, Pippins R, Habtemariam A, Gitterman A. FDA’s 21st Century Cures Act Guidance Documents Clarify the Landscape for Digital Health Innovation. https://www.arnoldporter.com/en/perspectives/publications/2017/12/fdas-21st-century-cure-guidance-documents. Accessed March 22, 2021. Published 2017.
7. Kent J. FDA Updates Draft Guidance on Clinical Decision Support Software. https://healthitanalytics.com/news/fda-updates-draft-guidance-on-clinical-decision-support-software. Accessed March 22, 2021. Published 2019.
8. US Food & Drug Administration. Clinical Decision Support Software. Draft Guidance for Industry and Food and Drug Administration Staff. 2019. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-decision-support-software. Accessed March 22, 2021.
9. IMDRF SaMD Working Group. “Software as a Medical Device”: Possible Framework for Risk Categorization and Corresponding Considerations. http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-140918-samd-framework-risk-categorization-141013.pdf. Accessed March 22, 2021. Published 2014.
10. Carli D, Fahrni G, Bonnabry P, Lovis C. Quality of decision support in computerized provider order entry: systematic literature review. JMIR Med Inform 2018; 6(1):e3. doi: 10.2196/medinform.7170 [Crossref] [ Google Scholar]
11. Ancker JS, Edwards A, Nosal S. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med Inform DecisMak 2017; 17(1):36. doi: 10.1186/s12911-017-0430-8 [Crossref] [ Google Scholar]
12. Muylle KM, Gentens K, Dupont AG, Cornu P. Evaluation of an optimized context-aware clinical decision support system for drug-drug interaction screening. Int J Med Inform 2021; 148:104393. doi: 10.1016/j.ijmedinf.2021.104393 [Crossref] [ Google Scholar]
13. Evans B, Ossorio P. The challenge of regulating clinical decision support software after 21st century cures. Am J Law Med 2018; 44(2-3):237-251. doi: 10.1177/0098858818789418 [Crossref] [ Google Scholar]
14. Vasiljeva K, van Duren BH, Pandit H. Changing device regulations in the European Union: impact on research, innovation and clinical practice. Indian J Orthop 2020; 54(2):123-129. doi: 10.1007/s43465-019-00013-5 [Crossref] [ Google Scholar]
15. European Commission. Medical Devices - Topics of Interest: Notified Body. https://ec.europa.eu/health/md_topics-interest/notified_bodies_en. Accessed March 22, 2021. Published 2017.
16. BigMed.no. Clinical Decision Support Software Regulatory landscape in Europe from May 26th 2020. https://bigmed.no/assets/Reports/clinical_decision_support_software.pdf. Accessed March 22, 2021. Published 2020.
17. DigitalEurope. Reflection Paper on regulatory frameworks for digital health technologies in Europe. https://www.digitaleurope.org/wp/wp-content/uploads/2019/04/DIGITALEUROPE-Reflection-on-regulatory-frameworks-for-digital-health-technologies-in-Europe-.pdf. Accessed March 22, 2021.Published 2019.
18. European Commission. Bodies - Regulation (EU) 2017/745 on medical devices. https://ec.europa.eu/growth/tools-databases/nando/index.cfm?fuseaction=directive.notifiedbody&dir_id=34. Accessed March 22, 2021.
19. Medical Device Coordination Group (MDCG). Guidance on Qualification and Classification of Software in Regulation (EU) 2017/745 – MDR and Regulation (EU) 2017/746 – IVDR. https://ec.europa.eu/docsroom/documents/37581/attachments/1/translations/en/renditions/native. Accessed March 22, 2021. Published 2019.
20. US Food & Drug Administration. Product Classification. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPCD/classification.cfm. Accessed March 22, 2021. Published 2021.