Why AI Verification Is the Real Bottleneck in Pharmaceutical Drug Discovery

David Finkelshteyn: The tagline for my company is humanizing AI for care. We've talked about how healthcare needs to be emotionally ready before it can be technologically ready. How people feel safe, seen, and empowered is that change happens.

Chris Hutchins: Hundreds of thousands of dollars on the data that powers the technology. Why is verification, not model accuracy, the hardest problem that you actually have to solve?

David Finkelshteyn: But what gives us the ability to kind of search through the much wider grid of possible compounds and molecules and biomolks. So the whole idea of AI now is to be able to fail faster. So we can get faster to an actually correct result.

Chris Hutchins: Today's guest is David Finkelshteyn, CEO of Pivotal AI, where he focuses on building AI systems for pharmaceutical life sciences use cases that can actually be verified, defended, and trusted. We're gonna go beyond the hype today to talk about the real bottleneck in pharma AI verification. Not making models faster, but making their outputs defensible enough to survive regulators, audits, and real-world consequences. David, welcome to the Signal Room. We're in some pretty interesting times, and I know you're doing a lot of great work in the pharmaceutical industry right now, among other things that you're working on. But if you don't mind, I'd love to just kind of jump right in with some questions. I know I've got a lot to learn, and I'm sure our my listeners are going to be very interested in what you have to say today. So let's talk about pharma AI. When people talk about it, they focus on discovery speed more often than almost anything else. Why is verification, not model accuracy, the hardest problem that you actually have to solve?

David Finkelshteyn: Um yeah, thank you first of all. Thank you for having me here. Yeah, I'm I'm excited to excited to be here and to share some uh of my uh experience. It's always a pleasure to have a conversation with uh smart people. Uh so let me go get one for you. Oh, come on. So answering your question, I don't think is one of the more is more important than another. I think they tie together and the discovery itself just doesn't make sense without verification because we should be able to get something, a molecule, the drug, that will fit some requirement parameters. So it should be plausible in terms of like pharmacinetics, pharma dynamics, uh it should uh fulfill admin, many other parameters. Um so in this sense, how like yes, we discovered something, but what is it we discovered, right? And this this this our discovery should be tightened together with verification both uh uh in our computer. And then we want to get down to five, ten, fifty, hundred molecules, and we want to actually synthesize them in the lab and we want to test them. And like it's this this can be we can start we should start with uh purely in vitre tests because it's the cheapest one, and we this way we will cut off a lot of uncertainty. Because again, our model, regardless how good is it and how accurate is it, how model itself, how verified is it, we need a ground truth. And ground truth is like actual physical molecules being synthesized. Regardless if it's small molecules and big molecules. So here it's just generating molecules without verification just doesn't make sense.

Chris Hutchins: Right. I understand that. You know, you you raise an excellent point as well in terms of the realities of what what we're using now with AI. It's just it hasn't been available to us for very long. It does require a lot of due diligence, especially in the space that you're working in. I I I really uh appreciate your perspective on that. I think it's for me it's comforting. I I hope it is for others as well. Let's think about the outputs. Um so when you're when you're using the AI, what are the most common ways or the the between discovery and regulatory acceptance? Where do things really break down?

David Finkelshteyn: So there are many possible points on breaking down. And it's again, it doesn't come specifically for yeah. So most of the breakpoints, they are the same regardless of the way we discover the lead. So there is uh possible breakpoints when we trying to synthesize it, then when we're trying to test parameters and and um of the drug in vitro, then when we go from in vitro to in vivo, like in the living creatures. Unfortunately, we still need to do this, then when we select in a like clinical target group, then when we go to the human testing, so all these points are possible breakpoints. There are um specific for artificial intelligence, like for for this way of generation drugs, there are specific breakpoints for this way, and they are again it's it's just uh other side of the metal. Because on on one hand, what AI what gives us ability to kind of search through the much wider grid of possible compounds and molecules and biomolecules. So because before, let's say we can like we knew some number of molecules, and like as a humanity we were inventing them and we were making like steps, and now they seem very like little steps compared to what we can do now with all this computational power and uh machine learning power. Right. But so like we did already a lot of tests on all these compounds and molecules, all sorts of like testing again in vitro, in vivo, all sorts of types of research of these molecules. And but so and when we AI gives us ability to simulate experiments with something far, far away from what we already have, something looks completely similar, completely like different, unlike regard if it's if it's a small molecule again or a protein, we can use an AI, we can design like using alpha fold, let's say we can design a protein that we've never met in a human body or in an animal body, just something artificial, and and alpha fold will predict how it will fold in space, and this is great. However, the verification, when verification came into place, it's this unseen protein may cause some again like may cause our body not to accept it. It may cause an allergic reaction, and because it's not it doesn't look like anything we've seen before, we have no information, almost no information and guess about it. So we just have to make more tests, and again, this is a possible point of breakdown. Yeah. And this is I would say, yeah.

Chris Hutchins: The speed that you mentioned, I I think this is the piece that gets me excited, but I can see all when you're talking about validation, it also complicates matters. But but you're able to really look at a much broader set of um of information and simulate and model different things that would take you a very, very long time before having that capability. But then you gotta play catch up on the validation side. That that must be it's exciting to to know how far fast you can go, but you there's some work that you just can't shortcut. I imagine that's a bit of a a frustrating point as you're as you're learning learning and and developing new new compounds.

David Finkelshteyn: I mean, yeah, it depends on the perspective. It's it's it can be frustrated on one hand, and on another hand, knowing that we are like safer from I mean, I am as a person who have been working with machine learning models more than 10 years, I can tell you that sometimes they so much off, like they're completely off. So it's it's actually makes me feel safer knowing that our validation criteria hasn't been changed for AI specifically.

Chris Hutchins: That being said, at least you know that you you you can fail faster if that's one of the benefits, right?

David Finkelshteyn: Exactly. And this is very, very important because again, 90% like I don't remember exactly. Um I think around 90% or even more of all drugs that after they were selected as uh drug candidates. So this whole industry, right? Everyone knows that create a new molecule, the novel molecule, even generic, but specifically the know is like huge money, tons of amount of money. You can't just you know be a startup or just enthusiast and just create new molecules. You have you have to get tremendous amount of money. And big part of it is this huge failing rate. And the whole idea, and you cat you catch it up, like it's this the whole idea of AI now is to be able to fail in faster. So we can get faster to an actually correct result. Instead of like putting a lot of hours and and efforts and human hours into and work into something that will fail in months, now we will know it will pass even our machine learning threshold in a week.

Chris Hutchins: That's a massive impact. That's exciting because I think think about the broader uses uh of AI for clinical research as well. Like this is a really exciting time that we're entering into. If we thought the COVID vaccines came out quickly, that's probably just scratching the surface of what's going to be possible as we learn how to use this the AI capabilities uh more efficiently moving forward. I've heard the term black box way too often when they're talking about data AI in any industry. Why do those models fail specifically in pharmaceutical environments?

David Finkelshteyn: Uh what so what do you mean by failing pharmaceutical environments?

Chris Hutchins: So the You know the idea of trusting AI, not necessarily having enough visibility into exactly how it's deriving the outputs that you see. So w where is that really problematic in the work that you do?

David Finkelshteyn: Yeah, so this is um essence of machine learning modeling per se. So we have a kind of this trade-off. It's not like we there are workarounds that we can talk about more later, but the the rule of summons there is a trade-off. The more complex model, the the more complex the task, usually the more complex model you need. And the more complex the model, the less transparent it is. Because for us, transparency is something that we can understand, we can relate, we can grasp, right? Because like the easiest, the most simple machine learning model, what is it? It's uh just uh a line, right? Which uh I think uh every data science first lesson you have a graph and you have some dots going approximately this direction, and everyone being asked, okay guys, now tell me, predict where next dot's gonna be in this graph. And now everyone drawing this line and saying, Okay, it's going this direction. So congratulations, you just uh make made your first prediction as uh like as a machine learning model would do. So it's we understand where this machine learning would come from, right? It's very simple. But when and then like there is a larger of complexity, and at the end we have uh large language models, it's a neural network with very complex structure, with hundreds of billions of weights. And of course, when you look at it, like and you have input and output, there is no way a human can tell, okay, this I think I know what this will give me, right? And then yeah, so it's able it is able to solve very complex tasks because it the structure is so complex and flexible, and this makes it not uh explainable so much. So we can't understand. People can't not always can understand the reasoning behind um decid like decisions that machine learning models.

Chris Hutchins: That's such an important thing, too. Is you we we really need to make sure that we're we're thinking about how we're working with AI. We we have to have explainability and transparency, particularly when it comes to anything that's going to touch clinical care or it's gonna be some sort of a treatment that that a human being is going to be subjected to. Regulations are a little bit slow and coming in when it comes to this side of everything, but I don't think we have a whole lot of time left before that starts to ramp. It sounds I'm starting to hear a lot more about it now. Hopefully we can be uh proactive enough in the industry that we don't have things done to us, which is usually how regulation affects us. We've got people who are really not understanding the tech the technologies, they're not understanding the medicine, they're trying to make policy, and that's that's just it's a recipe for disaster, in my opinion, particularly in the space that we're talking about, just because um if you're dealing with it from a regulatory standpoint, you're thinking about it from the legal and risks lens. And the people that are having those conversations are certainly not trained scientists, they're not trained clinicians for the most part. No, there are a few, but very few of them are actually in regulatory roles. I really appreciate that, the context that you that you're giving it. What does it mean to actually verify an AI insight in in the in the context of drug development?

David Finkelshteyn: So the the closest thing to a verifying AI. So when we define defining the model, again, it we it's we we're getting back to what we discussed at the beginning, right? The model, the model output there are just some numbers. The context is what is it important for the model. So we should define clear several things. First, we should define the um the specific use cases, the frame within which we can apply this model, right? Because again, uh there are models that can predict uh train models that generally speaking can predict pretty good, let's call it everyone knows I think AlphaFold, right? AlphaFold can predict pretty good how molecules are folding in space, and they're really doing a good job there. We want, and I will give you an extreme example just on that. We want regardless of how good AlphaFold in predicting molecules looks in space, we won't rely on AlphaFold to predict a new drug against some kind of cancer. Just because it's not intended to do this. It wasn't trained to do this, it wasn't validated to do this, it's it's not the job for this model. So first thing is a specific use case. Second thing is a data, created data. It's very, very important that how we what kind of data we use to train and test the model. Um again, I I I I don't think it should be a guide for data scientists how exactly to do this, but high-level data should be should fit some statistical distribution, which it shouldn't have too much outliers, too many outliers. Um at least they should be again distributed. Well, when we test, when we're validating the model, we should avoid the same called data leakage. So it means we should really separate the data which we use to train the model and that we use them to validate the model. And often as a data scientist, we have an urge when we have uh new data, and sometimes it's very like exciting, new data, some new can give us new sets. We have an urge to retrain our model using this new data. So this is a very dangerous path because if we retrain this model using this new data, we should have another test set to validate this model again, because otherwise we may overfit our model, meaning it will show very good results in our train data, obviously, in our validation set, which is not purely validation anymore, and therefore we cannot say how well it will fit the pattern, not the specific noise in in our data set. So this is this is very important. Yeah, I'd say these are main things.

Chris Hutchins: You mentioned context and I want to kind of stay on that topic for a minute here. Because I think that's one of the things that I'm starting to find out that people really are not as diligent as they should be in in how they're using AI. And I think the risk that is now emerging because we've we've got two companies that are now entering the healthcare space from a consumer access standpoint. And I mean, I wonder if maybe you can talk a little bit about how important context is, even in a larger sense, because we're talking about people just interacting with Chat GPT Health or with Cloud Health. These are AI companies. The data that they're working with, it's because it's sitting in platforms that are considered to be administrative and not clinical, it changes the dynamics in terms of who's responsible for decision making. And if it's not involving a clinician and it's sitting on a platform like that, in most or most situations, we're not talking about liability even being a possibility. So missing context, I think, is going to be something that we have to really pay attention to. Talk a little bit about that from your perspective and why it's such an important part of what you're doing.

David Finkelshteyn: So, yes. Um, the first thing I want to say here, I in my opinion, it's really important that people um uh trying to know more and figure out more about their own life, especially about their own health. So when you when you have an appointment to doctor or when it's when it's hurt and you like you do your own research before going and blindly trust someone. Even if this is a trained specialist, unfortunately we know that not everyone, uh even in in this area, um the same, right? There are better true specialists and and not so good. So it's always good to s have your own opinion and your and enable your critical thinking uh supported by some knowledge. That being said, of course, everyone, as I'm sure especially everyone who is listening to your podcast knows that Chat GPT, Jimina, I don't know, Claude, all of them, they they hallucinate it. And they do this on a daily basis, pretty often, right? So again, my rule of thumb here is always always try to find the ground truth, the origin, reference. So, and again, like the LLMs they're not hallucinating because they are stupid or because they don't know. Actually, no, they because they don't know, they they hallucinate because they don't know. But if you give them enough context, it's a very powerful analytics tool. So they can extract knowledge from again, they can gather and extract knowledge from different sources, different types of references. So what I suggest, the golden pass, I think, is to whenever you want to uh first thing, you whenever you want to do research, do your search, but when you use an LLM, request it to work with a real source, and of course, again the the golden standard scientific article. So if you want to research something about any type of symptoms or or disease, just ask it to give you a bunch of references and give you a suggestion based on it. And the second, of course, all right after this, go to the doctor. Have your knowledge, gather your knowledge, and go to the doctor. You're not trained enough, neither me nor you, no one trained uh uh trained enough unless you're a specialist. So do your research and go to the doctor.

Chris Hutchins: That is perfectly sad, and I think people really need to hear that message. We we probably should put that on repeat and turn the volume up, honestly, because the the context issue is from from my standpoint, just because of of my own experiences, I can't remember things that happened 25 years ago that actually might be entirely relevant for the condition that I'm that I'm dealing with right now. And if I can't remember it, the model's certainly not going to have that information available. So that the context has got to be something that people think about. And it's only like you said, it's not hallucinating because uh of anything except for really the lack of access to information. And there's a lot of things that go into that. And I think the things that I get concerned about are symptoms that for all by themselves, they might be low risk and there'd be no red flags set off. But with proper context, if it happens to be relevant that two years ago I had a really difficult time healing my broken leg because I had a wound, that kind of information in current context might mean something completely different than just the symptoms that you're trying to figure out what to do about now. That's just uh in my own thinking. I mean, I know that they're they're working through uh and deploying the doctor's version and uh you know a pay a patient's version, but again, still the context is important. I am on medication for my blood pressure. The model may not know that. The model might just say I've got high blood pressure and alerts, and I actually go to the physician and I'm taking up time uh that I don't need to be taking, which means the physician may not be treating someone that really needs treatment because I'm in there and I don't need to be. So I think there's like a lot of different sides to this, but in the context is really, really critical.

David Finkelshteyn: Again, because uh everyone who has uh internet and uh twenty dollars in the pocket, uh have access to like great analytics slash knowledge base, analytics tool slash knowledge base. It's a good, it's a it's a big trend for patients to own their own data. Right. Meaning when you go to the doctor, right? They you went there, they gave you some, they made a diagnosis, they may rate some rust some tests, and then they say you some tell you something, give one paper where it says like what they did and and and what the doctor did and you a very organized person if one year later you have this paper somewhere. It's a rare case actually. But but the clinician the clinic the the the the the hospital they always have your data they gather it they store it because they have to and for you to own this data to be able to again use the whole context for maybe LLM tool or another specialist to look at it this is a a big trend I'm seeing now in in the software development in this area and and in healthcare in general.

Chris Hutchins: Yeah this it's gonna be an interesting year because we started right out of the gate and we're not even halfway through well I guess maybe we are about just halfway through today, halfway through January. Yeah it's it's gonna be interesting to see where where things go from here. I want to kind of get into some things around data integrity and and data quality because that's that's one of the other issues that we're we're dealing with and you know you talk about the the absence of data from my standpoint that's the scariest kind of bias that we could be dealing with. Talk a little bit about you know how you how do you manage when you're trying to determine where to focus how difficult is it to validate whether you actually have all the right inputs, the right data sources and you you've got sufficient data integrity and quality that you can actually trust it and start to move forward?

David Finkelshteyn: Yeah it's a great question. I say this is uh it's actually the biggest bottleneck in in AI in drug design development in healthcare and uh AI and in general in all data science tasks. So it's um really like now the data is is like a gold everyone is running around trying to get more data generate more data and this is another problem synthetic data but again it's um it's a like big topic. Yes you you need to very carefully s get select get and select sources of your information curate the data uh make sure that it the the origin the sources of the this data are trustworthy and then you need to maintain integrity indeed and uh it sounds easy but actually it's not because there are a lot of errors happen along the way from from where you find the data to when it comes into the model to train it and all the things usually related with humans of human error the good thing that the again in theory the path to the integrity is pretty straightforward you need you just need your system fully traceable audible and you you have to log everything again it sounds easy it's not in reality but this is like this is how you do this is how you ensure integrity of your data make sure in every step along the way of your data pipeline you can always look backwards and and in every step you you you you know what happened with your data how it was transformed what what happened how it was like brushed and selected and curated so this this is the way to again preserve the integrity.

Chris Hutchins: Right. Yeah it's such an important factor and I I think that just thinking about the context again like when it when it comes to you know clinical care facts change and when you're you've trained your models to look at things a certain way you've got all kinds of historical data there. It may be actually just fine but the reality is things do change and if the model's not aware of what's changed then it's going to give you outputs that are going to take you in a direction you don't want to go. I think the real important thing for us to remember is when you're talking about healthcare, it's called the practice of medicine for a very specific reason. It's an evolving science it's not static. And unfortunately a lot of the regulatory agencies are they they treat it like it is static and that's how things are measured. Whatever the composite is that you're measured against to determine whether your quality is good or not you're either incentivized or penalized. It's based on a composite that no human being will ever look like and it's just a really important thing especially when we're talking about using uh LLMs as a consumer remember you are unique. The model doesn't know who you are and it doesn't care and it's certainly not going to have any compassion for you. So you know you got to just be aware of what it really is. And the incomplete risk, you know, from terms of having not enough data or not the right data it's the same issue that a clinician has now when they look at us in the exam room, they're wondering what don't they know and is it important? AI is not going to solve that problem. It may help to accelerate the processing of information and eliminate a few more risks or even diagnostics, but it's still not going to be able to give you the the precision that everyone wants. Yeah I think what I would also say to what you mentioned earlier in terms of how people should be thinking about it, you also need to remember when you when you're putting your PHI into a tool like that, you're also eliminating your own protections by doing so. People really need to be very, very careful how they think about using these things and just remember to your point, do your research but go see your doctor. It's just it's really important to do that. So let's talk about what I kind of touched on there and it's maybe the uncomfortable part of what we're dealing with. What are the weaknesses that are most often surfacing during audits or regulatory reviews or legal scrutiny? Because I mean obviously you guys have to think a lot about that when you're dealing with research and experimentation to actually get to some of the exciting breakthroughs that you get to so again about breakthrough specific um uh like problems specifically for drugs designed with help of AI uh or in general for I think it's across the board because I mean in in both in both perspectives you know I it's still a it's still an issue that's got to be uh you know thought about and and weighed it's just the the pace and I don't want to overuse the term black box but to regulators uh you know a lot of this stuff is gonna seem like a black box because they already have a language barrier in that they don't understand the scientific language that's used half the time or probably more than half the time.

David Finkelshteyn: Yeah again for regulator like okay there are two kinds um if you talk about if we're talking about AI in let's say drug development and drug design there are two we can divide our AI tools or approaches by two types. First type is when we and this is where we where we work in mostly uh as pivotal and what we were talking during this podcast mostly generating new leads so new new candidates for drugs right so this and for usually for the regulators for the authorities for FDA EMA they don't care much how we came up with this drug with this uh new molecule because anyway we're going to undergo all the same tests that other deno molecules going through regardless of the way we came up with these molecules right so it again it's um maybe someday down the road we will be able to like convincing be more convincing with the reasoning how we came up with this but again at this point it doesn't really matter how do you how do you came up with this molecule that you want to test it's just going through the standard pass. Now another type of machine learning tools or AI tools uh that comes into the pharmaceutical area these are more decision makers and they not really in the designing your drugs right uh they more in the analyzing information for example uh the biggest uh example of it uh it's uh approved by EMA let me look at I don't uh remember the name exactly uh of this uh this company this tool give me one second I will look them up yeah so aim uh uh NASH this is uh essentially a SS a uh AI assistant for histology scoring in uh mesh rails so it's uh aim I approved tool that help makes a decision and this is a completely different thing right though this is essentially became a part of validation chain and again we don't have because people don't trust AI and for good it's it's good they don't trust the uh right this that we don't have a lot of at all I think I uh this is the only one of its own kind that essentially allowed into the uh as a decision maker without human overview too much. So we just just not there yet.

Chris Hutchins: Right. As we're kind of getting to towards the end here, I want to maybe take a take a look out in the into the future a little bit. First we have talked about hallucinations, but what would you tell people that they should be thinking about and in terms of how can you prevent them in in the in the work that you're in the pharmaceutical industry in particular because I'm sure there's a lot of folks out there that are wrestling with how to use it and how not to use it you know but the hallucination piece of it I think is something to be very interesting to understand where you think that needs to go.

David Finkelshteyn: So again it really depends on what you want to do with this. In general again the rule of thumb if you want LLM to hallucinate less give it more context. The more context the more data you will fit uh LLM the less it will hallucinate it essentially think about think uh about LLM as a very powerful uh search slash analytics tool so just don't ask it to invent something ask it to analyze and work with the data and context you gave it.

Chris Hutchins: Beautifully said I don't know that that can be overstated that that's amazing. As you looking ahead do you think uh verification is gonna be really become a real competitive advantage rather than a constraint?

David Finkelshteyn: Yes I see now and it's not like again it's in a different stage of drug design the chain is very long from idea to actual drug in the pharmacy the the the chain is tremendous so and many steps in this chain that involves a lot of documentation about a lot of paper papers filled and this is where I take over now taking over now because as we all know works with documentations filling papers like based on your input fill some standardized templates papers from authorities this is driving it right it it doing it great and there are all sorts of kind of tools emerging now to taking over this task and this is great because again this is something that we shouldn't like there weren't any harm from this because they cannot spoof any evidence or push something some drugs that will be toxic or harmful in any way just there is a bureaucracy threshold and now we have a tool to fight with this bureaucracy at least at some portion of it. So this is where we see our acceleration I think again it's uh the seeing how fast we evolve in this industry and how like pivot every year in in this industry and other industries but yeah it's a prediction is a very hard thing to do. I'm not a good machine learning model is a good prediction what what I can say I think that again first uh we will uh try to do something with low-hanging fruits which are working with documentations uh and maybe AI will let us to reduce the amount of efforts we have to put into the bureaucracy and the in the red tape and this will lead us and like data scientists and biochemists and like other scientists and smart people in pharma to put their mind more into the essence of it into the creation of creation of new drugs. Getting back to our very first topic we may be able to shorten this loop of designing and verification and now we there are loops where we design in drugs in AI and then we send candidates to the automated robotic lab and this lab they generate molecules and they send them the oh sorry not generate that test the molecules send the result back. So it's kind of closing this loop and we need less and less human in it and this is very good. So the very good high quality verification loop will lead us to a faster drug invention and uh probably higher quality medicine.

Chris Hutchins: Exciting so just there's just so many things that weren't possible a decade ago that now are going to become reality. I think there's more groundbreaking discoveries coming in the cancer space. It wasn't that long ago the cancer diagnosis really came with a an unfortunate timeframe that you're dealing with because you it's it wasn't curable. But the things are evolving so quickly and the work that you you're doing in the pharmaceutical space incredibly important. David as well wrapping up if folks wanted to uh reach out to you to have a conversation because I'm I'm sure that you you've shared some things that'll folks will have some um curiosity about and want to get some some insight from you how how do they reach out to you? What's your preference?

David Finkelshteyn: So yeah you can reach out mainly my LinkedIn profile I hope link will be under the in the description right yes yeah and or uh there is a form on our website where you can uh ask a question or uh submit a request for project or consultation uh

Chris Hutchins: perfect yeah for for listeners i'll I'll make sure that his contact information in the website is uh in the show notes so David thank you so much for a really practical and grounded conversation I think you hit some themes on a on a high level that folks really need to be cognizant of context context context big big focus on that I really really appreciate it and I'm sure that uh the clinical folks that are listening out there appreciate that message too context is important but when you've done your research go see your doctor that's it for this episode of the Signal Room if today's conversation sparks something in you an idea a challenge or perspective worth amplifying I'd love to hear from you. Message me on LinkedIn or visit signalroom podcast dot com to explore being a guest on an upcoming episode. Until next time stay tuned stay curious and stay human