Artificial Intelligence Applied to clinical trials: opportunities and challenges
Scott Askin,1,2 Denis Burkhalter,1,2 Gilda Calado,1,3 and Samar El Dakrouni1,4
Author information Article notes Copyright and License information PMC Disclaimer
Associated Data
Abstract
Background
Clinical Trials (CTs) remain the foundation of safe and effective drug development. Given the evolving data-driven and personalized medicine approach in healthcare, it is imperative for companies and regulators to utilize tailored Artificial Intelligence (AI) solutions that enable expeditious and streamlined clinical research. In this paper, we identified opportunities, challenges, and potential implications of AI in CTs.
Methods
Following an extensive search in relevant databases and websites, we gathered publications tackling the use of AI and Machine Learning (ML) in CTs from the past 5 years in the US and Europe, including Regulatory Authorities’ documents.
Results
Documented applications of AI commonly concern the oncology field and are mostly being applied in the area of recruitment. Main opportunities discussed aim to create efficiencies across CT activities, including the ability to reduce sample sizes, improve enrollment and conduct faster, more optimized adaptive CTs. While AI is an area of enthusiastic development, the identified challenges are ethical in nature and relate to data availability, standards, and most importantly, lack of regulatory guidance hindering the acceptance of AI tools in drug development. However, future implications are significant and are anticipated to improve the probability of success, reduce trial burden and overall, speed up research and regulatory approval.
Conclusion
The use of AI in CTs is in its relative infancy; however, it is a fast-evolving field. As regulators provide more guidance on the acceptability of AI in specific areas, we anticipate the scope of use to broaden and the volume of implementation to increase rapidly.
Keywords: Artificial Intelligence (AI), Machine learning (ML), Clinical trials (CT), Opportunities, Challenges, Implications
Introduction
Evidence generated by CTs is widely accepted and likely to remain the gold standard for development of safe and effective drugs, despite the long-standing acknowledgment of the great investment and high risks involved for pharmaceutical companies [1, 2].
With AI being recognized as a pathway towards sustainable and optimized drug development, multiple applications in CTs are being discussed and begin to be explored in practice. This is boosted by the growth and expansion of randomized trials providing medical research with large and complex volumes of categorized and uncategorized clinical, molecular and imaging data. While data availability is critical for data-driven and personalized medicine trends, generating actionable insights from the available information requires use of comprehensive AI models, developed and trained with appropriate datasets, to effectively expedite and streamline the various activities within drug research [2, 3].
A range of opportunities are already identified in literature, starting with AI’s contribution to discovery in areas where return on investment might not support profitability (rare diseases, targeted therapies). In addition, anticipated increases in efficiency of patient recruitment and protocol design are suggested to improve chances of trial success, while monitoring of patients and analysis using AI may have the potential to positively impact measurement and interpretation of results [2–5].
The objective of this research was to identify and synthetize from literature, the opportunities for AI in CTs, the challenges potentially holding it back, and to identify future implications as AI implementation becomes the “norm”. As companies worldwide are considering which AI applications may offer the most benefit and begin early stages of piloting, regulators try to keep up with the pace and examine where regulations are needed. As such, this review may guide companies’ decision to invest in the integration of AI into their development programs. In addition, with such a broad array of applications and unclear sense of where industry may collectively be heading, it could help determine where regulation will be most impactful.
In this paper, the broader term of Artificial Intelligence (AI) and the general technique Machine Learning (ML) was used, instead of specific methods described in literature, as the purpose was to identify opportunities, challenges, and implications, rather than technically describe how solutions were achieved.
Methods
To identify relevant English-only publications a search was performed in PubMed, SCOPUS, International Pharmaceutical Abstracts and Google Scholar databases using the broad terms “artificial intelligence” or “machine learning”, with the term “clinical trials”. Exclusion terms referring to AI/ML applications outside of CTs were used to reduce the extensive list of unrelated results (e.g., “clinical practice”, “surgery”, “diagnosis”, “treatment”).
To identify relevant Regulatory documents the websites of the European Medicines Agency (EMA), the European Commission (EC) and the Food and Drug Administration (FDA) were searched using the same keywords.
Searches were performed between 7th October 2021 and 14th October 2021, with results being downloaded from source databases and websites and combined into a single excel file. To ensure that research was performed on state-of-the-art AI, any publications prior to 2017, as well as those not relevant to EU or US and any duplicates were removed. This ensured that the most recent and in-scope publications remained. Finally, the results were manually condensed based first on Title, followed by the Abstract content, with publications and/or documents not referring to applications of AI or ML in the context of CTs considered out of scope and excluded from this review.
Forty-eight publications were reviewed and categorized based on the research activity where AI was applied: Pre-clinical research, Design, Recruitment, Conduct and Analysis. Table 1 provides definitions for these category labels [6]. Additionally, the therapeutic area(s) (TAs) referred in the publication (if any) were accounted for. Nine documents extracted from Regulatory Authorities websites were categorized as Regulatory documents.
Table 1
Definitions of research activities considered for categorization of papers based on use of AI
| Pre-clinical research | Early use of AI in pre-clinical research, impacting subsequent CTs. |
| Design | Use of AI enabling prediction of outcomes and disease progression to shape or improve Design of CTs. |
| Recruitment | Use of AI in Recruitment, which includes Enrollment, defined as the identification of eligible participants and onboarding them into suitable CTs. |
| Conduct | Use of AI in Conduct refers to the period following a participant’s enrollment into the trial, up to the trial database lock, prior to statistical analysis. |
| Analysis | Use of AI in Analysis relates to activities performed by statisticians after a trial has achieved database lock, as part of statistical analysis for the trial. |
Figure 1, provides a further summary the process described above.
Flow Diagram of Papers Researched
Results
Our research indicates that the potential applications of AI across CTs is broad, however recruitment is clearly a key area of interest. Over 50% of the papers reviewed address recruitment (30), which is over double that of trial design (14) and nearly 3 times the number of papers discussing analysis (10), which were also commonly identified. Figure 2 shows the number of publications referring to AI applications, per CT activity.
Number of papers referring to AI applications, per categorized CT activity
This graph represents the application of AI across the categories of CT activities defined, as discussed within the publications reviewed. Out of a total of 48 papers that were in scope, 38 papers described application of AI to a single activity of a clinical trial, five papers described two activities, three papers described three activities, and the remaining two papers described four activities
In addition, Oncology is the most prominently discussed Therapeutic Area (TA). Out of the 22 papers which describe the application of AI within a specific TA, 50% discuss oncology (11). This is again over double that of the next most reviewed TA of Neurology (5), followed by Cardiovascular (3). Figure 3 shows the number of publications referring to AI applications per TA.
Number of papers referring to AI applications per therapeutic area [6]
This graph represents the application of AI across therapeutic areas (TA), as discussed in the publications reviewed. Out of a total of 48 papers that were in scope, 26 papers did not describe a specific TA. The distribution of TAs across the remaining 22 papers is represented in this graph, which include three papers that described two TAs within the same paper
Further analysis of the literature allowed identification of a broad array of opportunities for AI, with a clear emphasis however on tools that create efficiencies, regardless of the CT activity where it is applied. For example, AI tools used to inform CT design can result in reduction of the number of trial participants and trial length. As highlighted, recruitment is an area of great interest, with AI tools performing automated eligibility analysis, matching potential participants to trials, and simplifying trial searching capabilities. During trial conduct, AI-based sensors and other wearable devices improve patient monitoring. Finally, AI tools facilitate more comprehensive statistical analysis and tackle the challenging issues of missing data and missing visits, especially critical during the time of the Covid-19 public health emergency.
It is noted from the literature review, that fewer challenges were highlighted compared to opportunities. This may suggest that researchers perceive AI as offering great potential that is worth pursuing, despite the extensive work still needed to overcome significant challenges commonly identified. Namely, the lack of robust, standardized, and complete datasets, and the collaborative efforts required to build these data repositories. Of note, challenges about validation and model interpretation were also highlighted across CT activities.
Implications of AI in CTs are perceived very positively. Implementation of AI tools will speed up clinical development by increasing the probability of trial success and regulatory approval, use adaptive protocols will increase diversity of data, while reducing trial burden on both patients and sites.
Figure 4, provides a summary of the key opportunities, challenges and implications of AI, per activity of CT.
Summary of the key opportunities, challenges and implications of AI, per research activity of a clinical trial
Finally, the review of relevant regulatory documents indicates that AI is welcomed by Regulators as an innovative discipline with broad applications and disruptive potential but requiring strategic planning for implementation. Although there is record of international collaboration through a horizon scanning initiative covering AI, it is noted that EMA and FDA are approaching AI integration in healthcare differently. In the EU, there is an ongoing effort to develop a broad regulatory framework for AI. Whilst EMA’s harmonized guidance for AI use in the pharmaceutical sector and in CTs is not yet available, our research found that national initiatives may be filling that gap. In the US, FDA seems focused in the qualification of AI tools based in their classification as Software as Medical Device (SaMD), without specific acknowledgment for its potential use in CTs. As such, most FDA documents were found to be out of scope of this review and are not discussed.
Discussion
The identified opportunities and challenges, as well as the future consequences of AI integration are discussed below, summarized by key CT activities or as Regulatory relevant documentation.
Pre-clinical research
New target discovery and toxicity prediction
AI can contribute to address unmet medical needs by enhancing and accelerating identification of new molecular targets (genes or proteins). Access to large pharmacokinetics (PK) and pharmacodynamics (PD) datasets, from previous preclinical and clinical research (including from failed trials), is needed to develop and train effective and reliable algorithms that generate new stable molecules with real treatment potential. Lack of published PK/PD data, for competitive or proprietary reasons, is a significant hurdle to achieve full potential of AI in new drug discovery [4, 7].
Several AI methodologies for safety prediction are described. In fact, software is available to predict drug toxicity based on target information. Efficient toxicity predictions have the potential to replace in vitro and animal models as the traditional pre-clinical approach [8]. Additionally, the models can be used as risk-management and prioritization tools in development pipelines, by providing early indication of high-risk compounds flagged with significant safety concerns [4].
As in other AI fields, model interpretation can be challenging, especially given the high level of uncertainty in early phases of research. Understanding model features and underlying biological mechanisms is key for interpretability and confidence in predictions [4].
Design
Predicting patient outcomes in CTs
Prediction of clinical outcomes is essential to the advent of precision medicine and to inform trial design by eliminating the statistical variability of general populations. In fact, AI can be used to simulate data to detect more efficient statistical outcome measures [9]. One report suggests that using an AI algorithm to predict participant outcomes and to identify those most likely to progress fast and reach endpoints sooner, could lead to shorter duration trials [10]. In addition, by analyzing Electronic Medical Records, AI also provides an opportunity to predict likelihood of CT drop out [1]. Rather than excluding potential drop outs, efforts have been made in the cardiovascular TA to target these specific enrollees and provide them additional education to encourage longer participation [1]. Such tools can reduce overall sample sizes and therefore less participants are needed for the trial [1].
Another report demonstrated that ML prediction models allowed reduction of cancer mortality by 15–25% across several CTs [11]. Such ML algorithms, that enable clinical outcome predictions stratified by environmental and genetic attributes, can be created from large biologic databases correlating drug-related predictive biomarkers from interventional trials with progression-free and overall survival data, arranged by molecular profiles of tumors [11]. A further study analyzing data from Non–Small Cell Lung Cancer trials tested the validity of biomarker status (along with other more complex drivers) in predicting tumor response and survival rates using ML tumor growth models [12].
These AI tools can be implemented to enhance drug selection and adapt investigational drugs to the histology of a specific cancer thus increasing survival rate. Advancements are still expected, however with ML models including comprehensive data from multi-omic features, there is the potential to shift the treatment paradigm and redefine the way precision trials are designed and recruited for.
Predicting probability of trial success
There is evidence of ML algorithms supporting early detection and prognosis of disease, thus improving overall CT success [9]. Beyond clinical expectations, AI can be wielded in the early phases of clinical research to predict molecular features, target sensitivity, bioavailability and toxicity [7, 13], as well as to reduce later stage trial failure, and thus help design PhII/PhIII trials that are more likely to transition to regulatory approval. The impact on human and financial resources, as well as protection of participant’s safety and public perception of CTs, is irrefutable [3, 7].
ML models informed with CT design and patient characteristic datasets can be used, not only to predict regulatory approval, but also to estimate the probability of success in phase transitions [14]. Understanding the factors influencing success and failure per phase (mainly complexity of the protocol, selection of clinical endpoints, of the interventional arm and of the eligibility criteria) impacts the trial design in current and subsequent phases [3, 12]. Developing similar models based on risk scores of side effects or lack of efficacy available in the literature might also help design trials that better safeguard wellbeing and safety of participants.
Finally, within the Oncology area, AI is being applied to build In-Silico trials, which use clinical data to build simulated cohorts that model treatment effectiveness [15]. These also offer the potential to reduce later stage development failures by identifying better responders [15]. Still, the challenge remains that there is a lack of good quality, curated and complete datasets available, which often limits the potential of AI application [15].
Reshaping clinical trial design
AI solutions can enable faster and more accurate hypothesis generation and analysis to enhance our understanding of disease evolution, as well as to improve drug discovery, cohort composition, monitoring, adherence, and endpoint selection [2, 3]. In brief, improved outcomes are observed after implementation of AI methodologies in the design. For example, suitability of cohort composition is improved via protocol enrichment and biomarker verification. Nevertheless, collaborative effort is needed to build common protocols for collection, archive and organization of large datasets, thereby mitigating error in AI output [2].
In addition, well-designed AI tools, that have access to enough good data, could be used to predict disease progression within a virtual control arm [10]. This could allow a placebo arm to be replaced by a fully virtual arm consisting of synthetic data only [10]. Several benefits are anticipated including reduced budgets, reduced site and patient burden and potentially faster CTs. However, validation of virtual control arms based on training datasets built alongside existing CTs requires significant investment of time and resources. Synthetic control arms also have the potential to eliminate ethical concerns regarding placebo control groups and to encourage participants otherwise not willing to risk randomization to placebo [2].
It is a long journey to build the infrastructure and multidisciplinary expertise to support such novel technologies. Ethical considerations also remain a barrier, especially considering that protection against misuse of health information remains a challenge [3].
Recruitment
Among the general hurdles that exist in the set up and the conduct of a CT, recruiting participants remains a critical challenge. The impact of poor recruitment is known to be immense in terms of time and associated financial cost to the trial [16, 17]. Such recruitment challenges are due to complex protocols, lack of awareness of the trial, emotional fear of participation and often just a lack of interest to participate [16, 17].
Inclusion and exclusion criteria are also becoming more complex, making it difficult to recruit the right patient, who must meet required selection criteria to avoid potential confounders or misclassifications [18, 19].
Across therapeutic areas it has been highlighted that AI tools can combine data such as demographic, laboratory, imaging, and other -omics data, to match patients with those complex inclusion criteria, ensuring suitability to recruit [10, 16, 17, 20].
There is further opportunity for automated trial recommendation, meaning that AI could enhance patient selection by providing information to a broader cross section of potential trial participants via public CT platforms [17]. By using large scale datasets from the metabolic area it has been shown that AI tools are also supportive of fairer trial access, which can be considered an important implication [21]. Such approaches could be used for awareness in clinical trial searching and contribute to matching engines, as demonstrated in a HIV study [22].
Table 2 further describes examples of AI tools applied to Recruitment across a variety of TAs.
Table 2
Examples of AI applications within Recruitment
| Opportunity | Therapeutic Area | AI application |
|---|---|---|
| Patient selection and access fairness | Oncology and Cardiovascular | • Oncology:• Facilitation of cohort selection (e.g., AI technology applied to medical records to ameliorate recruitment and identification of suitable patients) [20, 44]• An AI-enabled clinical decision support system (CDSS) -based on natural language processing of cancer specific values and ML methods- to accurately identify eligible subjects with a high degree of sensitivity and specificity during a retrospective review of four breast cancer focused trials [20, 45]• Cardiovascular: AI/ML-based fairness metrics established for the purpose of equity in trial access [21] |
| Biomarker refinement | Neurology (Alzheimer) and Amyotrophic Lateral Sclerosis (ALS)) | • In Alzheimer disease: an AI classifier was optimized to detect asymptomatic cases for CT recruitment (otherwise not identified using the biomarker amyloid plague) [19]• In ALS: It has been shown that a robust ML survival model includes a broader approach to patient inclusion in CTs, by identifying patients that could have still benefitted from a trial despite originally being excluded [18] |
| Large scale analytics to support trial matching search engine | Infectious diseases (HIV) | • Large public database of interventional trials developed using AI, to support a search engine for a trial matching system to be used by HIV patients [22] |
An underlying basis for effective contribution of these AI tools in recruitment is implementation of standardized language for eligibility criteria allowing system interoperability. The tool must be able to read and understand the input, in order to enable its intended use [23, 24]. Therefore it is suggested that it can be beneficial to combine structured data with insights obtained from natural language processing of patient reports to supplement information for eligibility screening [25]. Some recent tools have already shown promising results in terms of system extraction, for example a clinicaltrials.gov database summarized the eligibility criteria from over 350,000 trials [26, 27].
Another opportunity is related to the high operational burden in pivotal CTs, leading Sponsors to maintain a short list of preferred investigators based on their expertise and performance on previous CTs. AI could provide a fast and facilitated approach to internal Investigator ranking, which would speed up site initiation and therefore positively impact recruitment [28].
Conduct
Use of digital health technologies (DHTs) in trial conduct
With the inclusion of automated data collection tools and by developing novel digital biomarkers that rely on AI algorithms to interpret data and transform it into usable insights, near real-time access using wearable devices and sensors can be provided to investigational sites, to obtain visualizations of a participant’s condition [29]. Improving the safety oversight of trial participants, especially those with life-threatening or debilitating conditions, is a clear advantage facilitated by faster access to actionable insights.
Other opportunities, reported as having been applied in psychiatric and neurological disorders, extend to adherence of Investigational Medicinal Product (IMP), which is a major challenge in CTs. Whilst other technologies are available to track when participants have opened their IMP, to presumably take a dose, evidence suggests these technologies may not provide accurate data [30]. This is where AI can offer improved methods to monitor and confirm IMP intake. A video capture device with a built in AI Algorithm can be used to more reliably confirm when a participant has taken their medication. Such a tool is suggested as being the only way of confirming study drug adherence outside of a member of the site staff physically watching the participant dosing, which is often not a feasible alternative [30, 31].
Analysis and workflow management of medical images
The use of AI to streamline the review and supplement the analysis of medical images is a well-recognized opportunity for trial conduct activities. AI algorithms can support the automated annotation of important markers, which would normally be derived manually by experts [10, 32]. AI could also improve workflows for imaging review by using automated image classification tools and speeding up expert’s reading time [29].
A significant challenge in this area is the very time-consuming and labor-intensive effort of building up a repository of good, standardized images, needed to train algorithms [32].
Analysis
Determining effect heterogeneity
Whilst investigations show that rarely is there a perfect homogeneous treatment effect, identifying effect heterogeneity is a well-known problem for CTs statisticians [33]. AI applications were trained using cardiovascular datasets, to interrogate CT data and identify subgroups that showed differing treatment effects, as well as to identify key risk factors and fast-responders in sub-populations [16, 33]. Such tools present an opportunity for more comprehensive analyses and better insights for drug developers. However, the challenge of regulatory acceptance requires that researchers find ways to fully validate results provided by these new models [33].
Imputing missing data and handling missing study visits
During the COVID-19 pandemic, many CTs were impacted due to the reassignment of clinical site resources, difficulties in travelling to a site, and participants catching the virus or having to quarantine because of it. These issues could lead to missing data and delays of study visits, with impact on statistical analysis [34]. ML can be employed to impute data that is missing, and also to infer a participant’s condition when visits have been delayed beyond the protocol defined windows [34, 35].
Automation to support analysis
Our research also highlights that AI tools could support the automation of data extraction into statistical analysis tools to reduce the need for manual effort and associated human error [5]. As seen in other AI applications, challenges in this area concern the work needed to develop and validate such algorithms [5].
Regulatory documents
In the EU, initiatives concerning AI are taking place at different levels. EC is developing a general framework and governance model for AI, founded in excellence and trust. Key priorities are infrastructure, expertise, and respect for principles (e.g., accuracy and supervision, security and privacy, transparency, diversity and non-discrimination) to safeguard ethical aspects and EU fundamental rights [36].
The proposal for EU Regulation, known as Artificial Intelligence Act, establishes conformity assessment for high-risk AI (including SaMD) covering risk management, data governance, automatic record-keeping, human interface, and cybersecurity requirements [37]. Implementation of this regulation requires the creation of an EU-shared database and full access to data sets in pre- and post-marketing phases, both particularly challenging [37].
Use of AI in development and approval is included in EMA’s Regulatory Science Strategy to 2025. Pharmaceutical sector stakeholders prioritize the building of regulatory framework and guidelines on AI validation and assessment, in collaboration with academia and expert centers [38]. Whilst EMA’s harmonized guidance is not yet available, Member States are taking national initiative. For example, the Italian Regulatory Agency’s (AIFA) Guide for submission of CTs identifies as major risks of AI/ML incorporation the safety of CT participants and validity of clinical data collected to support approval. To address these, AIFA requires submission of benefit-risk assessment showing that AI integration is useful and valuable, does not have safety implications and the algorithm is reliable in its contribution for evidence generation [39].
In the US, FDA’s Digital Health Center of Excellence (DHCoE) has released guidance on development and certification of SaMD and launched the pre-certification pilot program for software, both applicable to AI/ML solutions used in CT context if classified as SaMD [40]. In addition, recently published Good Machine Learning Practices for MD address challenges highlighted by several publications reviewed (e.g., representativeness of data sets to mitigate bias; continued assessment through model surveillance and risk management during real-life training of the model) [41].
From a global cooperation perspective, it is worth noting that the International Coalition of Medicines Regulatory Authorities (ICMRA) identified AI in the top three innovation topics challenging current regulation. Problematic areas are acceptance of AI-generated data and algorithm evolution, requiring prior scientific advice. Sponsors planning to use AI tools in CTs should fully disclose algorithms for evaluation and justify advantages over traditional methods in recruitment, diagnosis, monitoring of disease progression or end-point measurement [42]. This encouragement for early advice is an opportunity for sponsors to increase approval chances, share expertise and also influence future regulatory guidance on digital health solutions.
Other challenges foreseen by ICMRA concern trial implementation (e.g., more complex informed consents; usability of AI tools by participants/investigators) and building of AI expertise in regulatory agencies, ethical committees and data and safety monitoring boards, adequate to support the evaluations [42].
Lastly, the World Health Organization (WHO) highlights ethical use and governance as primary concerns on AI use in healthcare. Ethical challenges in CT context include the use and management of confidential data collected with consent; assurance of data inclusiveness to avoid bias and inequality in output; access and control over algorithms limited by private commercial ownership and associated legal frameworks. WHO recommended principles should guide construction of regulatory frameworks and healthcare practices that integrate AI [43].
Strengths and Limitations
Our review provides information on AI applications specifically categorized by CT activity and related therapeutic area(s), which uncovers insights that expand beyond existing literature reviews. It was focused on publications published between 2017 and October 2021, relevant only to US and EU, which might have excluded more recent publications on AI use in globally relevant markets. Studies discussing certification of AI interventions classified as SaMD were considered out of scope, despite these might have provided additional information on requirements for acceptance of AI tools in CTs. Finally, our research does not provide information on outcomes of AI in CTs from a comparative perspective, which is an area for further research.
Conclusion
Integration of AI in CTs is a promising and expanding field, and many articles suggest AI may be the key to overcome the current status quo in drug development and pave the way for a new paradigm of sustainable medical research. Efforts devoted to assessing the prospective use of AI are evidence that the Industry is looking forward to unlocking AI’s full potential in conducting more successful and cost-effective trials.
The approach to integration of AI in drug development and approval is broad and covers all phases of a drug’s lifecycle. However, despite existing publication of AI-related reflection papers and strategic action plans by Regulators, there is still a lack of specific and detailed regulatory guidance focusing on the use of AI within CTs.
Healthcare, being one of the most highly regulated and risk-averse businesses, is expected to witness a slow but safe adaptation to digital transformation globally. Therefore, despite the enthusiastic reports, there is still extensive work needed, including definition of proper ethical and regulatory frameworks.
In the near future, all stakeholders are encouraged to share learnings and join forces to build a robust, ethical and patient-centric approach to a standardized integration of AI in research. As such, sponsors, investigators, and regulators considering normalization of AI use in CTs need to build together the missing infrastructure and expertise to ensure patients ultimately remain safe and protected.
Author Contribution
The authors contributed equally towards the following areas: research question formulation, abstract writing, literature search, data extraction and tabulation, drafting paper, references organization, critical revision of paper, English editing, and tables and figures creation.
Writing sections were assigned to one responsible author, followed by critical review by all other authors. All authors substantively revised the work. The author(s) read and approved the final manuscript.
Funding
This paper was developed as part of the Regulatory Affairs and Health Policy Master’s Degree program at the Massachusetts College of Pharmacy and Health Sciences (MCPHS), Boston Massachusetts. Funding was not provided by the University for the development of the paper; however, the authors were sponsored by their employer to voluntarily undertake the master’s program.
With thanks to Rania A. Mekary, Associate Professor at MCPHS University, for her guidance and support in the development of this paper.
Data Availability
Not Applicable.
Code Availability
Not Applicable.
Declarations
Ethics approval
Not Applicable.
Consent to participate
Not Applicable.
Consent for publication
Not Applicable.
Conflict of Interest
The authors are all employees sponsored by their employer to undertake the Regulatory Affairs and Health Policy Master’s Degree program, for which this paper was developed. Two authors are shareholders of the company for which they are employed.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
1. C K, KW J, WW T. How artificial intelligence could redefine clinical trials in cardiovascular medicine: lessons learned from oncology. Personalized Med. 2019;16(2):87–92. doi: 10.2217/PME-2018-0130. [PubMed] [CrossRef] [Google Scholar]
2. S H, P S, B A, J H. Artificial Intelligence for Clinical Trial Design.Trends in pharmacological sciences. 2019;40(8):577–591. doi:10.1016/J.TIPS.2019.05.005 [PubMed]
3. Delso G, Cirillo D, Kaggie JD, Valencia A, Metser U, Veit-Haibach P. How to design AI-Driven clinical trials in Nuclear Medicine. Semin Nucl Med. 2021;51(2):112–9. doi: 10.1053/J.SEMNUCLMED.2020.09.003. [PubMed] [CrossRef] [Google Scholar]
4. C G, N M. The missing pieces of Artificial Intelligence in Medicine. Trends Pharmacol Sci. 2019;40(8):555–64. doi: 10.1016/J.TIPS.2019.06.001. [PubMed] [CrossRef] [Google Scholar]
5. M AG. Creating efficiencies in the extraction of data from randomized trials: a prospective evaluation of a machine learning and text mining tool. BMC Med Res Methodol. 2021;21(1). 10.1186/S12874-021-01354-2. [PMC free article] [PubMed]
6. Clinical Data Interchange Standards Consortium (CDISC). Therapeutic Areas by Disease Area | CDISC. Accessed October 25., 2021. https://www.cdisc.org/standards/therapeutic-areas/disease-area
7. A Z, Q V, TI O. Will Artificial Intelligence for Drug Discovery Impact Clinical Pharmacology?Clinical pharmacology and therapeutics. 2020;107(4):780–785. doi:10.1002/CPT.1795 [PMC free article] [PubMed]
8. AO B, A Y. Artificial Intelligence for Drug Toxicity and Safety. Trends Pharmacol Sci. 2019;40(9):624–35. doi: 10.1016/J.TIPS.2019.07.005. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
9. Sangari N, Qu Y. A Comparative Study on Machine Learning Algorithms for Predicting Breast Cancer Prognosis in Improving Clinical Trials. In: Proceedings – 2020 International Conference on Computational Science and Computational Intelligence, CSCI 2020.; 2020:813–818. doi:10.1109/CSCI51800.2020.00152
10. CS L, AY L. How Artificial Intelligence Can Transform Randomized Controlled Trials.Translational vision science & technology. 2020;9(2). doi:10.1167/TVST.9.2.9 [PMC free article] [PubMed]
11. AV S, IF AB, SB T. Machine learning model to predict oncologic outcomes for drugs in randomized clinical trials. Int J Cancer. 2020;147(9):2537–49. doi: 10.1002/IJC.33240. [PubMed] [CrossRef] [Google Scholar]
12. KW S, CH SK. Machine-learning and stochastic Tumor Growth Models for Predicting Outcomes in patients with Advanced Non-Small-Cell Lung Cancer. JCO Clin cancer Inf. 2019;3(3):1–11. doi: 10.1200/CCI.19.00046. [PubMed] [CrossRef] [Google Scholar]
13. F Z. Artificial intelligence in drug design. Sci China Life Sci. 2018;61(10):1191–204. doi: 10.1007/S11427-018-9342-2. [PubMed] [CrossRef] [Google Scholar]
14. F F. Key indicators of phase transition for clinical trials through machine learning. Drug Discovery Today. 2020;25(2):414–21. doi: 10.1016/J.DRUDIS.2019.12.014. [PubMed] [CrossRef] [Google Scholar]
15. FK LK, O G, C K, RB H. The case for AI-driven cancer clinical trials – the efficacy arm in silico. Biochim et Biophys acta Reviews cancer. 2021;1876(1). 10.1016/J.BBCAN.2021.188572. [PMC free article] [PubMed]
16. EH W. The role of machine learning in clinical research: transforming the future of evidence generation. Trials. 2021;22(1). 10.1186/S13063-021-05489-X. [PMC free article] [PubMed]
17. LM JVSA, JC BMRPH. Using supervised machine learning classifiers to estimate likelihood of participating in clinical trials of a de-identified version of ResearchMatch. J Clin translational Sci. 2020;5(1). 10.1017/CTS.2020.535. [PMC free article] [PubMed]
18. JD DB. Development and validation of a machine-learning ALS survival model lacking vital capacity (VC-Free) for use in clinical trials during the COVID-19 pandemic. Amyotroph lateral Scler frontotemporal degeneration. 2021;22(sup1):22–32. doi: 10.1080/21678421.2021.1924207. [PubMed] [CrossRef] [Google Scholar]
19. M A, S E, G G, et al. Reduction of recruitment costs in preclinical AD trials: validation of automatic pre-screening algorithm for brain amyloidosis.Statistical methods in medical research. 2020;29(1):151–164. doi:10.1177/0962280218823036 [PubMed]
20. JT B. Artificial Intelligence Tool for optimizing eligibility screening for clinical trials in a large Community Cancer Center. JCO Clin cancer Inf. 2020;4(4):50–9. doi: 10.1200/CCI.19.00079. [PubMed] [CrossRef] [Google Scholar]
21. O MQ, MA C, DM F. Quantifying representativeness in randomized clinical trials using machine learning fairness metrics. JAMIA open. 2021;4(3). 10.1093/JAMIAOPEN/OOAB077. [PMC free article] [PubMed]
22. K Z DDF. Automated classification of eligibility criteria in clinical trials to facilitate patient-trial matching for specific patient populations. J Am Med Inf Association: JAMIA. 2017;24(4):781–7. doi: 10.1093/JAMIA/OCW176. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
23. L C, Y G, X J, et al. Clinical trial cohort selection based on multi-level rule-based natural language processing system.Journal of the American Medical Informatics Association: JAMIA. 2019;26(11):1218–1226. doi:10.1093/JAMIA/OCZ109 [PMC free article] [PubMed]
24. VGV V. Hybrid bag of approaches to characterize selection criteria for cohort identification. J Am Med Inf Association: JAMIA. 2019;26(11):1172–80. doi: 10.1093/JAMIA/OCZ079. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
25. Shivade C, Hebert C, Regan K, Fosler-Lussier E, Lai AM. Automatic data source identification for clinical trial eligibility criteria resolution. AMIA Annual Symposium Proceedings. 2016;2016:1149. Accessed October 17, 2021. http://dx.doi.org/pmc/articles/PMC5333255/ [PMC free article] [PubMed]
26. S TK. EliIE: an open-source information extraction system for clinical trial eligibility criteria. J Am Med Inf Association: JAMIA. 2017;24(6). 10.1093/JAMIA/OCX019. [PMC free article] [PubMed]
27. H L, Y C, A B, Y S, C W. A knowledge base of clinical trial eligibility criteria.Journal of biomedical informatics. 2021;117. doi:10.1016/J.JBI.2021.103771 [PMC free article] [PubMed]
28. D JG. Optimizing clinical trials recruitment via deep learning. J Am Med Inf Association: JAMIA. 2019;26(11):1195–202. doi: 10.1093/JAMIA/OCZ064. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
29. Weissler EH, Naumann T, Andersson T, et al. The role of machine learning in clinical research: transforming the future of evidence generation. Trials. 2021;22(1). 10.1186/s13063-021-05489-x. [PMC free article] [PubMed]
30. K G, Z S. Assessing the scope and predictors of intentional dose non-adherence in clinical trials. Therapeutic Innov Regul Sci. 2020;54(6):1330–8. doi: 10.1007/S43441-020-00155-X. [PubMed] [CrossRef] [Google Scholar]
31. V K, A A, L Z, et al. Accuracy of machine learning-based prediction of medication adherence in clinical research.Psychiatry research. 2020;294. doi:10.1016/J.PSYCHRES.2020.113558 [PubMed]
32. Mayorga-Ruiz I, Jiménez-Pastor A, Fos-Guarinos B, López-González R, García-Castro F, Alberich-Bayarri Á. The role of AI in clinical trials. Artificial Intelligence in Medical Imaging: Opportunities, applications and risks. Published online January. 2019;29:231–43. doi: 10.1007/978-3-319-94878-2_16. [CrossRef] [Google Scholar]
33. BA G. Using machine learning to identify Heterogeneous Effects in Randomized clinical trials-moving beyond the forest plot and into the forest. JAMA Netw open. 2019;2(3). 10.1001/JAMANETWORKOPEN.2019.0004. [PubMed]
34. WR Z IB. Machine learning for clinical trials in the era of COVID-19. Stat Biopharm Res. 2020;12(4):506–17. doi: 10.1080/19466315.2020.1797867. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
35. N Z PM. Does including machine learning predictions in ALS clinical trial analysis improve statistical power? Ann Clin Transl Neurol. 2020;7(10):1756–65. doi: 10.1002/ACN3.51140. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
36. European Commission. White Paper on Artificial Intelligence: A European Approach to Excellence and Trust. ; 2020. Accessed November 24, 2021. https://ec.europa.eu/info/sites/default/files/commission-white-paper-artificial-intelligence-feb2020_en.pdf
37. European Commission. The Artificial Intelligence Act. European Commission. Published online 2021. Accessed November 24, 2021. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52021PC0206&from=EN
38. European Commission’s Heads of Medicines Agency. Joint HMA/EMA workshop on artificial intelligence in medicines regulation | European Medicines Agency. Published 2021. Accessed October 17., 2021. https://www.ema.europa.eu/en/events/joint-hmaema-workshop-artificial-intelligence-medicines-regulation
39. Dri DA, Agricola E, di Marzo M, Massella M, Verrelli NM. Clinical Trials Office Guide to the Submission of a Request for Authorisation of a Clinical Trial Involving the Use of Artificial Intelligence (AI) or Machine Learning (ML) Systems AIFA Italian Medicines Agency AIFA Working Group Drafting the Guide.; 2021. Accessed November 24, 2021. https://www.aifa.gov.it/documents/20142/871583/Guide_CT_AI_ML_v_1.0_date_24.05.2021_EN.pdf
40. FDA. Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan. ; 2021. Accessed November 24, 2021. https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device
41. Health Canada FDA. MHRA. Good Machine Learning Practice for Medical Device Development: Guiding Principles.; 2021. Accessed November 24, 2021. https://www.fda.gov/medical-devices/software-medical-device-samd/good-machine-learning-practice-medical-device-development-guiding-principles
42. ICMRA. ICMRA Informal Innovation Network Horizon Scanning Assessment Report-Artificial Intelligence. Published online 2021. doi:10.1038/s41573-019-0024-5
43. World Health Organization (WHO). Ethics and Governance of Artificial Intelligence for Health: WHO Guidance. ; 2021. Accessed November 24, 2021. https://www.who.int/publications/i/item/9789240029200
44. D CW KG. Improving clinical trial participant prescreening with Artificial Intelligence (AI): a comparison of the results of AI-Assisted vs standard methods in 3 oncology trials. Therapeutic Innov Regul Sci. 2020;54(1):69–74. doi: 10.1007/S43441-019-00030-4. [PubMed] [CrossRef] [Google Scholar]
45. Haddad T, Helgeson JM, Pomerleau KE, et al. Accuracy of an artificial intelligence system for cancer clinical trial eligibility screening: retrospective pilot study. JMIR Med Inf. 2021;9(3). 10.2196/27767. [PMC free article] [PubMed]