The Role of Artificial Intelligence in Clinical Trial Design and Research with Dr. ElZarrad

Dr. Roach: Welcome to “Q&A with FDA,” from the FDA’s Division of Drug Information, where we aim to answer some of the most frequently asked questions that we’ve received from the public.

My name is Dr. Sara Roach and today we will be discussing the role of Artificial Intelligence, or AI, in clinical trial design.

AI, including machine learning, is all gaining traction in clinical research, changing the clinical trial landscape, and is increasingly being integrated in areas where FDA is actively engaged, including clinical trial design, digital health technologies, and real-world data analytics.

Today we are joined by Dr. Khair ElZarrad, Director of the Office of Medical Policy within FDA’s Center for Drug Evaluation and Research to discuss recent advances and use of technology in clinical trial design.

Dr. ElZarrad leads the development, coordination, and implementation of medical policy programs and strategic initiatives and works to enhance policies and improve drug development and regulatory review processes. He was also recently awarded the 2023 Arthur Flemming Award, which honors outstanding employees.

Good afternoon Dr. ElZarrad, congratulations on the award! We are so glad that you could join us!

Dr. ElZarrad: Good afternoon Sara, my pleasure. Great to be with you today!

Dr. Roach: Can you set the stage for our audience by describing AI and machine learning?

Dr. ElZarrad: Sure absolutely. So generally AI and machine learning can be described as machine-based systems that can, for a given set obviously, for human-defined objectives, make predictions, recommendations, or decisions. Those AI systems use machine- and human-based inputs to perceive real and virtual environments. So they, they can abstract such perceptions into models in an automated manner. They use model inference typically to formulate options for information or action.

You know, machine learning, specifically, I would say, which is the most used form of AI that we’ve seen in drug development, is a subset of AI, and it employs a set of technologies or techniques that can be used to train AI algorithms to improve performance at the task, at the specific task, based on data that’s available.

Dr. Roach: It might not be well known that FDA has already received hundreds of submissions that reference use of AI. Can you talk more about that?

Dr. ElZarrad: Yeah sure absolutely, it’s a very exciting area actually! So over the last few years, FDA has seen a rapid growth in the number of submissions that reference AI. Obviously, when an application references AI that doesn’t tell you how deep or complex the use of AI. Nonetheless, we’ve seen a rapid rise in those applications. The last data I have, actually, from 2016 to today, approximately 300 submissions we’ve received that reference AI use. These submissions transverse the landscape of drug development, all the way from discovery to clinical research, you highlighted clinical trials for example in the beginning, but also to post-market safety surveillance and to even manufacturing, you know advanced manufacturing specifically.

We are also working to better understand how the use of AI in any one specific setting relates to participants safety and the reliability of study results. I call those like the two pillars that we have to pay attention to, the safety and the reliability of results.

The use of AI, including to facilitate data collection, this combined with robust information management, advanced computing abilities that we’ve been seeing increasingly in the recent year, are really transforming the way drugs are developed and used, so we are very excited about this area.

Dr. Roach: One of the most significant applications of AI in drug development is in efforts to streamline and advance clinical research. Can you tell us more about how these technologies are being applied in clinical trials?

Dr. ElZarrad: Yeah absolutely. Let me first say that one of our key objectives at the FDA is to really advance the modernization of clinical trials. Clinical trials themselves are cornerstone for evidence generation. We rely on them, and we must continue to make them more agile, inclusive, and innovative. You know and you can say multiple other areas where we can innovate around clinical trials.

So over the past few years, I hope everyone took note of our guidances, for example, on decentralized clinical trials or what we call DCTs, the digital health technologies or DHTs, and modern good clinical practice guidelines, among many other areas that hopefully will, will initiate a new generation if you would, of clinical trials both modernizing on the design front and the conduct front.

AI itself is being utilized – actually it’s being used to analyze a vast amount of data, and this is data from both clinical trials and observational studies. And its use is really to help make inferences regarding the safety and effectiveness of the drug being evaluated. AI also has the potential to inform the design and efficiency of clinical trials, including in decentralized clinical trials. And you see also a potential for AI use in trials incorporating the use of real-world data, another really main topic for us. For example, there is a great potential for AI uses in the extraction and organization of data from electronic health records and medical claims, as well as other data sources that tend to be valuable, sometimes unstructured. So, there’s a great potential for AI to assist us across the board here.

Dr. Roach: For those unfamiliar with DCTs – in these trials, some or all of the clinical trial’s activities occur at locations other than a traditional clinical trial site. These alternate locations can include the participant’s home, a local health care facility, or a nearby laboratory that incorporate DHTs, which are systems that capture health care information for the clinical trials directly from individuals. Can you elaborate on how decentralized clinical trials are currently incorporating digital health technologies?

Dr. ElZarrad: Yeah absolutely. It’s important to mention first, two points. Decentralized designs and the use of digital health technology hold a great potential in streamlining clinical trials in general, but also in expanding the reach of trials, and reducing the burden on participants. We really hope those designs and those tools can help us reach rural communities, for example.

That said, DCTs and DHTs are not silver bullets on themselves, and should be evaluated to see if they fit, if they are fit, for purpose, while considering the total context of the clinical trial, the type of the intervention, and the population involved among other factors. Take for example, that we should not really assume what patient’s preference would be. You know some patients may like to have a healthcare provider come to their homes. Others may not. So I think we should do our due diligence in understanding how those designs can and could be incorporated in the context of the trial.

Another point is that many DHTs are portable instruments obviously, such as you know activity trackers, glucose monitors, blood pressure monitors, spirometers, and they contain advances such as electronic sensors, computing platforms and information technology. So we have to consider all of that as we think about incorporating those tools.

DHTs are also, could include interactive mobile applications, where participants take for example, can rate their quality of life. They can rate pain, depression, daily function, or even perform tests of functional performance such as cognition, coordination, vision and among many other factors.

Many DHTs may be worn as you know, like implanted, ingested, or placed in the environment. And this is a really critical point because when we think of DHTs a lot of times we think of wearable trackers. This is, goes far beyond that. And placing those DHTs in a specific environment and allowing real time collection of data from trial participants in their homes and their locations, which, as you mentioned, remote to the clinical trial sites, can help gather real-world data relating to patient health status or the delivery of health care. You know, it’s practically, rather than getting a segmented view of patients behavior or input, you’re going to get a bit more of a holistic picture.

One area that I‘m keen on is how we can use technology to assist in the recruitment and retention of trial participants, as well as explaining trial processes to those participating in a more user-friendly and meaningful way. You know, one of the key reasons for failure in clinical trials is lack of recruitment. So can we utilize those tools to do better in this area is something we’re very much interested in.

Dr. Roach: This could really benefit clinical trials and drug development.

Dr. ElZarrad: Yeah, absolutely. You know, like, there are many examples, you know. Take DHTs – may enable continuous or frequent measurements of clinical features, or measurements of novel clinical features even. Those could, not be captured usually in traditional study visit because you go to the site. Typically, you capture data at the point. DHTs can enable much more really, in capturing more comprehensive data in that sense.

Those are typically state-of-the-art digital health technologies that we’re seeing increasingly. They allow participants of themselves to partake in the clinical trial remotely, so you can for example see how they are a natural fit for decentralized clinical trials, you know. Like for example, you can really reach communities by utilizing those tools that have never been reached before any clinical trials. Typically, those communities also have a huge burden of the disease.

We still though, we have to need to avoid a little bit the flashiness of those new technologies. You know, that we call them this new tool. Everybody wants to be, you know, part of this innovation right? So, but we have to really avoid the flashiness behind this new technology and ask, first – Do they fit in the context of the trial. I can give you one example in a recent meeting with academic experts, they really highlighted to us that customization and incorporation of technologies may also lead to increased complexity. We tend to think of those tools as removing complexity. But we really have to think about the context in its totality.

Dr. Roach: Considering the tools, data, and analytics we have discussed so far, such as Digital Health Technologies and Real-World Data, how is AI being increasingly integrated in areas where FDA is actively engaged?

Dr. ElZarrad: Yeah, good question. Many DHTs are AI-enabled, either as you know, embedded algorithm within the DHT itself, or employed upon the data generated from the DHT after the data is collected. So the utility of AI can really be in multiple places here.

Take one example: AI has been used to predict the status of chronic disease and its response to treatment, or to identify novel characteristics of an underlying condition. AI can be utilized to analyze the large and diverse data sets. This is, can be generated from continuous monitoring for example, continuous monitoring of participants. And AI can also be used for a range of data cleaning and curation purposes, an area that’s really important in the context of real-world data. This could include, for example, identifying duplicate participation, imputation of missing data values, as well as the ability to harmonize controlled terminology across drug development program.

One other point is that AI has also been used to analyze high volumes of real-world data extracted from electronic health records, medical claims, and disease registries, among other sources. We are also seeing AI being used in creating EHR phenotype, or patient cohorts, based on health records and claims data.

Dr. Roach: Are there other applications of AI being actively explored that you can share?

Dr. ElZarrad: The use of AI in predictive modeling, or counterfactual simulation, and in silicone modeling, for example, to inform clinical trial designs is being actively explored, and we’re really excited about the potential there.

In addition, AI used to improve the conduct of clinical trials and augment operational efficiency is also being explored. As I mentioned, making clinical trials more agile is a critical aspect here.

We’ve seen AI being used to assist in recruitment, and is being really developed and used in, in, to more effectively connect individuals as part of the trials. This can be really, can involve mining vast amounts of data from diverse sources, as we mentioned before, but also including social media, medical literature, registries, and structured and unstructured data in electronic health records.

So I think key to all of this really is recognizing that the potential is huge, but context matters and thoughtfulness is really essential across the board.

Dr. Roach: It appears that AI may also help improve clinical trial diversity. Can you expand on how AI is impacting recruitment and the logistical aspects of clinical trial design?

Dr. ElZarrad: Yeah, absolutely. And diversity is a very critical aspect for us, and an important aspect. But it’s important to remember that diversity in clinical trials is really a complex issue and it will take multifaceted approach, multifaceted solutions, to help ensure that trial participants reflect the population that will ultimately take or use the intervention if approved.

AI can be used to help with site selection for example. Trial operational conduct could be optimized by utilizing AI algorithm, take, to help identify which sites have the greatest potential for successful recruitment to the trial.

Its use also can help enhance site selection, improve participant recruitment strategies, and support more targeted engagement initiatives.

AI has been explored already and used in part of a clinical investigation in the prediction of an individual participant’s clinical outcome based on baseline characteristics. This supports, for example, our enrichment strategy that we have a separate guidance on. And this enrichment strategy could aid in participant selection in clinical trials.

Dr. Roach: AI’s applications are quite transformational.

Dr. ElZarrad: Yeah absolutely. You know, we do at the FDA, we discuss and we recognize the potential for AI to enhance drug development in so many ways. And across the spectrum of drug development, you know I mentioned even manufacturing, and informing post-market safety as well.

Dr. Roach: What are some specific ways you are seeing AI transform the way we have traditionally approached clinical trial design?

Dr. ElZarrad: Yeah, the advancements have been, as you said, substantial, and the potential is really massive. Today, AI can be used to characterize and predict pharmacokinetic profiles, for example, after drug administration. And these kinds of models can really be used to optimize the dose or dosing even regimen in any selected study, you know. And this is really an important part of drug development.

AI can be used to monitor and improve adherence during the clinical trial. This can be through tools such as smartphone alerts and reminders, electronic tracking of medications, tracking of missed clinical visits. This is all can trigger potential non-adherence alerts, and hopefully provide us with the basic information to address that.

AI has already been used in clinical research to improve medication adherence, actually. This is specifically through the use of applications, like digital biomarkers, facial and vocal expression for example, to monitor adherence, remotely. I recall specifically one specific tool that is in development by academia and industry to track adherence in that form.

This technology has the potential to improve retention and participants’ access to relevant trial information. This can be done by enabling AI chatbots, for example, voice assistance, and intelligent search. So really, the potential, like, you can run through so many modalities of where the potential is really great here.

The last one I want to mention is that data from digital health technology and other systems can be used to develop participants profiles to potentially predict dropouts as well, and this would, could help in participants retention. So, if you know, somebody is likely to drop out of the trial, you can try to employ some methods to prevent that from happening.

Dr. Roach: AI has a role in clinical trial safety monitoring too. Can you expand on safety?

Dr. ElZarrad: Yeah sure. AI-enabled algorithms have the ability to detect clusters of signs and symptoms to identify potential safety signals, and that can be done in real time, which is again one of the areas that is really powerful. AI can be used to predict also adverse events in clinical trial participants. And this is an area we are definitely interested in exploring.

Dr. Roach: There are clearly numerous benefits to using these technologies, but what are some of the drawbacks or challenges?

Dr. ElZarrad: Yeah this is a great question. In, in recent years the integration of AI in drug development has gained significant traction as we discussed already. But this integration, coupled with continuous advancements of AI, not only holds the potential to accelerate the development of safe and effective drugs and enhance patient care, which is an important point. It also has to be considered very carefully. We’ve seen, for example, concurrent with this technology advancement, the use of AI in regards to submissions, have already increased. However, AI and drug development present, I would say, some unique challenges.

Take first, the variability in the quality and size and representativeness of data sets for training AI models. This can introduce bias and raise questions about the reliability of AI driven results. Responsible use of AI demands, truly, that the data used to develop these models are fit for purpose and fit for use. This is our concept we try to highlight and clarify. The fit for use really means that the data should be both relevant. You know, the data could include key elements and sufficient number of representative participants, and also that the data is reliable. And, and what we mean by reliable are factors such as accuracy, completeness, traceability.

Another important factor is because of the complexity of those computational and statistical methodologies behind the AI model, understanding how AI models are developed and how they arrive at their conclusion can be really difficult at times. And this may necessitate, or require us, to start thinking of new approaches around transparency.

Another factor is that the uncertainty in employed model may be difficult to interpret and explain or quantify at times, so that would require us to in a way try to figure out how to employ a risk-based approach. To look under the hood if you would, and how, how to do that in a reasonable and effective way.

And then finally, the last thing I would like to mention is that another challenge with some of those AI models is that the performance of those AI models could degrade over time, especially if you think of learning systems, right, where the data inputs are introduced, new data is acquired. And then the outputs may differ over time. So how do we make sure that we do not see that lag and performance. You know, I think it’s called in a, a data drift concept here.

Dr. Roach: How is FDA preparing to educate industry and to overcome these challenges?

Dr. ElZarrad: So internally, FDA has established a steering committee to provide advice, and this is advice on the general use or feasibility of DHTs and the implementation of DCTs. DHTs again, digital health technology, DCTs decentralized clinical trials. And as I mentioned, AI has been utilized across the board here. So we’re trying to provide engagement and also meaningful input on the use of those technologies and designs. To engage the public and receive outside opinions, we also are hosting multiple workshops, announcing demonstration projects, and the goal here is really to push and encourage mutual learning.

FDA recently had published a discussion paper, I think the title was “Using Artificial Intelligence and Machine Learning and the Development of Drugs and Biological Products,” and this is really aimed to spur discussion with the community. Obviously we have some expertise in the FDA. But we know a lot of the expertise lies outside of the FDA. We’re very interested in learning and making sure that whatever we develop when it comes to our regulations is responsive to the community where it encourages innovations, while at the same time safeguard, you know, patient safety.

This paper that I mentioned, we solicited specific feedback on specific challenges that we see. And it outlined that very nicely. And we’ve been receiving some great inputs from the community. So we’re very excited to really make sure we incorporate the thoughts and the, the future processes around, you know, policy development.

I just want to mention too, that we’re very interested in feedback, not only from pharmaceutical companies that are involved typically in drug development, but also on those who are developing AI algorithms from ethicists, from academia, from patients and patient groups, and also global counterparts. We realize this technology is really being developed across the world. And we want to, we want to do our due diligence by engaging globally in that sense.

Dr. Roach: In fact, our Small Business and Industry Assistance program, or SBIA program, recently discussed digital health technologies in our Clinical Investigator Training Course, and is always looking for ways to assist and educate industry, so this feedback is extremely important! A link to the presentations can be found by clicking on the episode webpage and navigating to the hyperlink.

Dr. ElZarrad, thank you so much for joining us today. Do you have any final thoughts that you would like to convey to our audience?

Dr. ElZarrad: Thanks again so much for the opportunity here. You know, as regulators, we’re really excited about the potential of AI and technological innovations in general. We do hope that such innovation will result in facilitating drug development and ultimately accelerating how safe and effective drugs get to those who need them in a quick fashion.

We plan on developing and adapting a flexible risk-based regulatory framework that will promote innovation and protect patient safety. You will see that not just for AI, but across our guidances and policies that will address technological innovations.

As we move further into integrating AI in drug development, we are committed to continuing to engage with all interested parties. We will share preliminary considerations, seek input, and encourage discussions on fostering the responsible use and deployment of these technologies.

As I mentioned the mutual learning is really critical for us, and we hope as we move forward collectively we shape this field in a responsible way. We do recognize that we need to learn from experts and experiences across sectors here. And as I mentioned, not just the standard traditional sectors. We really need to go beyond into the technology, into ethics and beyond. We are very excited to see how these innovations are being used and will continue to be used to accelerate the development of safe and effective drugs. And we definitely want to be part of the solution to continue this innovation as fast as we can. Thank you.

Dr. Roach: Thank you again, Dr. ElZarrad. As we’ve discussed today, and as with any innovation, AI creates opportunities along with new and unique challenges. Thanks for listening to “Q&A with FDA”. The full podcast and transcript of this recording is available at fda.gov/qawithfda. Many of our episodes offer continuing education credits for health care professionals, so be sure to visit the webpage for more details. If you are looking for additional learning or continuing education credit opportunities, including live and home study webinars, you’ll also want to check out fda.gov/CDERLearn and fda.gov/DDIWebinars. And if you have questions about this episode, or anything drug-related, email us at druginfo@fda.hhs.gov.

Resources

• Artificial Intelligence and Machine Learning (AI/ML) for Drug Development
• Digital Health Technologies (DHTs) for Drug Development
• The Evolving Role of Decentralized Clinical Trials and Digital Health Technologies
• Using Artificial Intelligence & Machine Learning in the Development of Drug & Biological Products – Discussion Paper and Request for Feedback
• Real-World Evidence
• FDA Clinical Investigator Training Course (CITC) 2022