Why AI Agents Are Healthcare's Next Biggest Security Risk

Chris Hutchins: There's a comforting story we tell about AI and security — that the same frontier tools defending the hospital are always a step ahead of whoever's attacking it. My guest today spent his career on the other side of that assumption. He'd tell you AI didn't break healthcare's data security — it exposed what was never really working, and it handed attackers a brand new surface to go after. Welcome to the Signal Room. I'm Chris Hutchins, and my guest today is Pranava Adduri, co-founder and CTO of Bedrock Data — a data security and governance platform built for the cloud, software-as-a-service, and AI era. Pranava was a founding engineer at Rubrik, where he cut his teeth on ransomware and backup recovery. He led data risk and compliance products at AWS, scaling a new line past $200 million in revenue, working directly with Fortune 500 CISOs. And he incubated modern data governance ideas as an entrepreneur in residence at Greylock. So when he talks about securing AI inside a hospital, it's from the engineer's seat, not the slide deck. Pranava, welcome to the Signal Room.

Pranava Adduri: Thank you very much. Great to be here today.

Chris Hutchins: Well, before we dig in too much, I'd love for you to just tell the people a little bit about who you are, and tell us a little bit about what Bedrock does.

Pranava Adduri: Absolutely. A little bit about me: we immigrated to the States when I was five. Grew up in California most of my life. And I think ever since I started working, I've always been in some flavor of data — whether it was scale-out data storage, whether it was scale-out data backup, or ransomware and data security for companies that have the most critical data sets. It's always been some form of data, including Bedrock now as well. And a little bit about what Bedrock does: Bedrock is a data security platform, built for enterprises of the largest scales to understand their data — especially as they're collecting this data in this AI era more than ever. As that data is rapidly expanding, being able to make sense of the data that people are creating and transacting with, and then being able to make sure that that data is being used correctly — that it's not being used in a way that puts the company at risk, or the customer's companies at risk. And so where Bedrock really sits is at the intersection of data and how that data is being used by the business on a day-to-day basis. And now, especially as AI agents come online and agents are transacting on behalf of humans, how those agents are making use of that data as well — we sit at the center of all of that. And the way we like to think about it is, if you can ensure that the correct guardrails are in place, then the company can actually move fast and realize the value of AI properly. So that's what we're doing.

Chris Hutchins: Well, you make it sound easy, but I'm certain that it's not. You spent your early career on ransomware and backup recovery at Rubrik. What changed about the problem once the target was a hospital instead of an ordinary business?

Pranava Adduri: The short answer is people's lives are at stake. And the problem gets very real very fast. Anyone doing a ransom will do reconnaissance, figuring out where it will hurt the target the most, such that they have the maximum likelihood of getting a successful ransom payment. And in healthcare, unfortunately, those are operationally critical systems that may administer actual medical devices that are giving life-saving support. It could be the medical systems. Anything that gets in the way of a patient getting care is something that can affect the outcome of a patient. And unfortunately, there have been instances where, because of ransomware attacks, people have been impacted very severely. So the short answer to your question is: lives are at stake when an adversary decides to target a hospital like this.

Chris Hutchins: Right. I don't know what makes people think of these kinds of nefarious things, but it's alarming, and I'm really glad there are people like you who are on the front line making sure we're constantly monitoring and hardening that surface. So let's bring this up to today for a leader who's just starting to pay attention. People might know about gen AI, they may know it exists, but not much beyond that. What part of the threat landscape are they most likely to be missing right now?

Pranava Adduri: When it comes to leaders who are exploring gen AI for the purposes of transforming their business or bringing operational efficiencies, I think it's easy for people to start seeing the light in the sense that — these are all the ways we can bring automation, these are all the backend processes that took forever, that someone had to do, that now a set of agents can go handle. The part that's easy to miss, though, is that you have these actors that are operating on instructions you've given them, but in terms of making sure they're aligned, and making sure they're acting in a way that exercises the judgment a human operator would — that's the part where it's not obvious all the ways these things can go rogue. It's not obvious all the ways that an agent might, in the service of trying to be helpful and do what you asked it to do, do things that you didn't want it to do. Like keeping the data somewhere it shouldn't be, because it was easier for it to work off of the data in that new location than to do it in a more secure way. At the end of the day, these agents are trying to get the job done, but the path that they take to get the job done — that's the part that's the unknown. That's the part that's hard to fathom for people. And so, to answer your question about what people might be missing, I think it's how the work gets done. That is something that is not obvious when you set these agents off to go do a task.

Chris Hutchins: Yeah, there are so many ways this is similar to other things we've gone through from a transformation standpoint. I remember the first time I used the Waze app on my phone. After the first couple of times it got me where I needed to go, all of a sudden I started to trust it. And then I went to a new city, things were a little more compressed, and I would start to miss turns because it wasn't moving that fast. It's just one of those scenarios where we're constantly seeing evolution and adoption of technologies, and it's at a pace where it's getting a little bit too fast for people to be able to keep up and understand what the risks are. I mean, there's an assumption that frontier AI capability stays on the defender's side. Is that actually true?

Pranava Adduri: I think that despite the safeguards that frontier labs try to put on these models, the consensus is that it's only a matter of time before open-source models also catch up in capabilities. And so while the frontier labs themselves can try to be responsible with respect to the controls they put, the vetting they put on these models to ensure that only defenders get the advantage — take Mythos, for example — while they can do that, while they're putting the effort into doing that, it's only a matter of time before the open-source community, and models that are more available to the public domain, catch up in that tier of capability. And once they do, the adversary will have just as powerful tools. So unfortunately, as much as one would want these advanced capabilities on the defensive side, we have to assume that our adversary too, in a matter of time, will have similar capabilities. The uptick in AI-assisted attacks can already be felt. If you look at IBM's data breach report, for the first time in a while it's not stolen identities that are the top cause for breaches — it's actually other tactics now that are more feasible for adversaries to leverage because of AI.

Chris Hutchins: It's interesting. I did some research a few months ago and was looking at the fraud and the cost of some of the social-media fraud that goes on. I really hadn't thought about it, but I discovered that the versions of this technology that we use, like a ChatGPT — they have their own version that they're using. They're constantly going back and forth: we're trying to increase our ability to detect, they're trying to subvert that. I had no idea. It raises an interesting question, because the advantage is not guaranteed at this point. So if both are getting better — the honest goal, we want to get better at not being breached. But if someone's determined to get to a hospital, people can really get hurt, and the data might actually get penetrated, or worse, it leaks out. What is really the honest goal that we have to think about there?

Pranava Adduri: To your point, I think that as these models get more advanced, the stark reality people have to accept is that the likelihood of getting breached is going to be higher than it was pre-AI. And this is in spite of all the investments people might be making to do better detection and response, better patching. Even with all that in place, I think the likelihood of a motivated adversary's ability to get in is now higher. So I think organizations have to start adopting the mentality: the likelihood of getting breached is higher — how do I get better at being breached?

Chris Hutchins: That's an interesting way to frame that. It sounds kind of backwards.

Pranava Adduri: It's a little bit of a neck-turner when I say it, but I think you have to assume that for a motivated adversary with all the tools at their disposal — and mind you, we're still in the very early arc of all of this; the evolutions of these models that we're going to see in the coming years are going to make what we have right now look like child's play — so we have to assume that the likelihood of getting breached is higher. And it's not all doom and gloom. If you assume that you will be breached and you take the directive of “how do I get better at being breached,” that changes the playbook. It changes the playbook in terms of: I need to take a risk-based approach. I need to figure out, if someone did get in, what are the most likely places they could get in, and what are the most valuable targets they might be going after. And if I work backwards from there, I can really start playing the book. The heuristic I like to think about is: start with the data that is most valuable to you. If this data were compromised, it would be company-ending. So what would the top five categories of data be, if you just sat down and did that exercise? Okay — next step. Where is that data sitting right now? If I know where that data is sitting, I can start there. Instead of doing an infinite search of all possible risks, I can start there and start modeling: well, if an adversary were to get to this, what are the roads they could take to get to this data? Okay, now let's focus on those roads. What are the entry points that would allow them to leverage those roads? And then you can systematically start hardening and layering the compensating controls that way. So for me, a good way to get better at getting breached is to start with what matters the most and work backwards from there.

Chris Hutchins: So — this wasn't obvious to me when I started to look at what you've been talking about. Maybe this is crazy, but if you know that's going to happen, and obviously you know one of the harder places, do you think about ways to essentially lead the witness, so to speak? Set some traps for them, take them off on a tangent way away from what's really important?

Pranava Adduri: In the sense of potentially having honeypots that you can deploy — that is one way to go about it. But I think honeypots are one part of a broader way of approaching the problem. Since we were talking about hospitals: think about key cards. Key cards lead to rooms in the hospital. Let's assume there are a lot of key cards. Well, not all key cards should be leading to the operating theaters. So if every key card can lead there, you should probably start with: what key cards can lead to these operating rooms, and do those people really need to have access to those rooms? Let's work backwards from there. In the ideal state, you're monitoring the most critical assets, you're monitoring the key cards that lead to them, and if any key cards end up accessing these operating rooms, you're able to immediately detect that and respond to it. The situation most organizations find themselves in right now is that, because there's so much data and so many identity grants, figuring out what data is most sensitive and what key cards lead to these sensitive rooms is itself a giant manual exercise. So imagine instead you have the ability to continuously keep up with the data, to understand what that data means to the business and the impact it has, and to continuously monitor who's getting access and how that data is being used. Then you can actually start to take control: these key cards shouldn't have this access anymore. In the ideal world, if there's a key card that's randomly dropped and an adversary finds it and scans it to get in, it leads to a storage closet where there's nothing of value. That's the ideal situation. It should not be leading to an operating room.

Chris Hutchins: Right. Well, it's a natural way — it helps me to understand and think about things differently. But what does this mean in terms of how AI gets access, and where the handoffs are happening — where the keys are, and they're moving around inside, and maybe we're not even aware of it? A lot of AI agents are getting granted broad access just to make them work. In a clinical setting, why is that a real problem? And what does governing an agent's access actually look like?

Pranava Adduri: Well, in order to make an agent useful — in order for an agent to give you time back in your day — the agent needs to be able to do useful things. And to do those useful things, it needs access. And it's exactly as you mentioned earlier, where you started using Waze for a while, and after a first couple of touchpoints where it got you to point B in a good amount of time, you started trusting it. The same thing happens here. The agent does a couple of jobs for you well, and you start trusting it, and you start giving it more access. That's how the access expands. The big difference between agents and humans with access is that agents don't have judgment. Agents have instructions they've been given — that they should do certain things and should not do certain things. But the judgment of how to evaluate a certain situation, and whether they should or should not take this path to get to a certain outcome — that's not guaranteed, and that's not there either. That's the fundamental gap. So in a hospital setting, if you're using agents to update records, to perform actions, the risk is that in the service of getting the goal done, it might take a set of intermediate steps that are not aligned with how a hospital should be conducting itself. It can go off path, even in the service of coming back to its original goal. I'll give you an example. We've seen instances — this is actually something a Fortune 500 was experiencing — where people tried to use their agents to build dashboards to see how certain metrics were performing. But to build that dashboard, the data in the source location wasn't in an ideal form. So the agent very simply created a temporary location in a shared space that a bunch of people had access to, and started copying the information there, because it was easier to use the data in that new combined location to build the dashboard. And yeah, it got the job done — but at the expense of making that data accessible to everyone else in the company. These are the types of judgment calls an agent wouldn't be able to make unless it had a scaffolding that basically told it, “you're about to go off the guardrails, don't do that.”

Chris Hutchins: Right. It's interesting — just doing some basic stuff that I've been playing around with, I see that kind of behavior. All of a sudden I discover it's gone off and done something I've either explicitly told it not to do, or that it never occurred to me it would do. Oftentimes, because we trust so easily, we forget that human beings make these things — so there are going to be some tendencies we have to be aware of. You've got a rule that clinical AI has to pass a test: does it actually give time back? I love that you say that, because I've spoken to so many clinicians who've been so frustrated with how we develop technology for them, because we've done the opposite of that. We've been taking time away from what they went to medical school for, which is to take care of people. Does security ever pass that test, or does it always work against it?

Pranava Adduri: I think there's no reason security doesn't have to pass that test. If you have a rule that says, in order for someone to truly do the secure thing, the secured path has to be easier than the default path — because people will ultimately choose the path of least resistance to get something done — then the onus is on leaders and security innovators to build solutions that are easy to use. The way I think about it is: you should still use AI to go do a certain task, but there should be a layer guiding the AI, a scaffolding that, if it's about to do something off the rails like keep the data in a shared location for ease of use, says “no, that's not allowed.” AI is very robust. If you say that's not allowed, it'll say, “I guess I can't do that, let me figure out what else I can do.” So it is a self-recovering system. It can handle you saying no to it, but you need that guidance. That is where good AI governance, good AI security, comes into play. Some system needs to be aware of the data, aware of the corporate policies, aware of the actions that are allowed, and it needs to work hand-in-hand with the agents to guide them. That is exactly the problem domain that we are thinking about as well.

Chris Hutchins: Yeah. So let's talk about where this all goes wrong quietly. I think we frequently get into a false sense of comfort and security — we've touched on it a little bit — but where do you see leaders getting a false sense of security and safety right now?

Pranava Adduri: I think there are a couple of areas. One is — when these models first came out, everyone was freaked out about sending data to third-party frontier labs. And they assumed, okay, well, Amazon hosts it on the Amazon Bedrock ecosystem — not to be confused with Bedrock Data — once Amazon hosts it, now it's within my corporate environment, so it's safe. In a way, yes, the model now resides in a VPC that you control. But at the end of the day, if there are agents getting powered off of that model, everything we talked about earlier is still a risk in terms of the judgment calls of these agents. There's maybe a gap where people assume, okay, we've brought the model within our four walls, so the big risk is over. That's like saying “I put in a firewall, so I'm good to go.” Not really. There's all the downstream — the model is the engine, but there are all the downstream agents and other workloads using that model that now have access. Getting a handle on what the blast radius is, what these things can do, and then reining that blast radius in, reducing it, making it so that if an adversary or an agent is doing something you're minimizing the harm it can do — and then also monitoring the agents. I'll give you an example of another Fortune 500 we're working with. They had an enormous data analytics environment — multi-petabyte. And because it's an analytics environment, by design a lot of people have to have access to it. There are a lot of key cards. So to reduce the likelihood that an adversary picking up a key card could get to something dangerous, and to get themselves ready for agentic operations, the exercise they went down was: one, what is all the sensitive, customer-identifiable information in here? Let's find that data, and first figure out why it's there. Is someone actually using it? If they're not, let's decommission that data — let's reduce that blast radius. And in the process, they're getting better at getting breached; they're reducing the footprint an adversary could get to. Then, what are the key cards leading to it? Who has key cards they're not even using? A great starting point to just reduce the key cards an adversary could pick up. And then, for the remaining key cards that need to be there and are actively being used — do these key cards have access to rooms the person's never using? If so, keep the key card, but reduce the grant so it can't get into that particular room. In this way, they've significantly hardened their environment. Now, all save for a couple of admins, everyone has a safe view of that data. So now agents are giving people time back, and they're comfortable letting it run in these environments, because they've reduced the scope of what these agents can do — even if it went off the walls, what it can do with the data is safe. So that's an example of putting it all together.

Chris Hutchins: Right. When we started seeing these data-visualization platforms come out and they kept leapfrogging each other — that still goes on — but the piece that was disturbing to me as a chief data officer was that every once in a while I would see a beautiful-looking dashboard, and the data was just wrong. And what worries me is the same caution I would give leaders: these things may look official. That's the danger of it. It can make an average Excel jockey look like a design wizard, and the prettier it is, the more they trust it, unfortunately. Now flip that over and look at the data access layer. I've always been bothered by the fact that I would have developers having full database-level access. Now, from a clinical system that sits on top of it, we have privileged account controls for that. But people dealing with a data architecture on the back end — I felt like there was always a level of exposure. From your experience, do you still see a lot of risk and exposure because that's just not understood?

Pranava Adduri: The short answer is yes. When we go into accounts, we do see a lot of privileged access. And the enemy tends to be — it's because at a certain point in time, that access was warranted, when the system was getting set up or during a certain maintenance. The problem is, afterwards, people forget to recycle that, forget to reduce it back down. So over time, access tends to grow. It doesn't decrease. The problem a lot of teams find themselves in is that, because these access and entitlements grow, when it does come time that this problem needs to be reckoned with, they have this forest of permissions, and it's not even clear what all the ways someone gets access to something are. So yes, absolutely. People grant the privilege so that work can get done, but because they don't necessarily go redact it after the fact, these things grow and become these giant scary forests over time. That's how people find themselves in these situations. And now is exactly when you want these agents to go do work for you. If you're letting that agent act as you, that's when it comes back to bite — because, again, these agents don't have your judgment. So if you set them out to do a task and they have all the grants that you have, that's when they can start creating these unintended side effects.

Chris Hutchins: Exactly. It's a whole other level of risk that I never anticipated seeing anything like. But it's always bothered me, because I couldn't protect my team members. A great example — in the healthcare sector, oftentimes there are these “break the glass” alerts that fire if you access a privileged account. A colleague of mine is an informaticist; he knew what he was doing. He was just doing some reconciliation, not paying attention to the names or the numbers — just making sure the fields matched up. And all of a sudden security showed up at his door and asked him what he was doing accessing one of the executive's records. It was a reality check for him right then and there. He said, okay, we need to make sure our team understands how all this stuff works, and we need to make sure we've got safeguards built in at the database level that prevent this kind of thing. Because at the very least, it's frightening to have a security person show up at your door when you're just doing your job. For me, it's the protection of the team that's having to work with this stuff that's always been a concern. So I really appreciate you laying this out, because I think leadership needs a better understanding and to think of it from that context — because no one wants to be the one that allows a breach, let alone the organization. For me, it's as much an investment to protect your team members as it is the data. If one CEO listening takes away a single thing from our conversation today, what should it be? And where can people find you and Bedrock? I'm quite sure there are going to be a lot of folks who have some interest in having a conversation with you, because you've raised some really good concerns.

Pranava Adduri: Well, if there's one thing to remember: these models are only going to get better. So the likelihood of an event occurring — whether it's a breach, whether it's an agent going off the rails — will only go up. And so, as the founder of Reflex Security said, in the phrase I was using earlier: learn to get better at being breached. I really liked that phrase he used. Assume something will happen — and then, how do you prepare for it? My advice is to start with the data and work backwards. Data is the ultimate anchoring function for taking a risk-based view of cyber. Start with the data, work backwards. That would be the one thing, if someone had to remember something: get better at getting breached, start with the data, and then work backwards to harden your environment.

Chris Hutchins: Pranava Adduri, co-founder and CTO of Bedrock Data. The line I'm taking with me is: the goal is never to be unbreachable, it's to get good at being breached. That's going to stay with me for a long time. It's not obvious for people who aren't living in the world you're living in — you've got to get good at being breached, so the day it happens isn't the day everything stops. Thank you so much for being on the Signal Room. I really appreciate your time today. And just briefly, let people know where to find you online if they want to reach out.

Pranava Adduri: Absolutely. We post regularly on LinkedIn with our learnings from the field and where we see the industry going. So find us on LinkedIn, and you can also find us at Bedrockdata.ai. We'd love to hear from you. And thank you as well, Chris, for the time.

Chris Hutchins: Thank you. And for our listeners, I'll make sure everything's in the show notes. If you want to follow up, you can — I'll leave a trail there for you to find it very easily, because we're at a time where we need this kind of expertise on speed dial. So again, thanks so much for being on the Signal Room. To my guests and listeners, I'm Chris Hutchins, and we'll be seeing you next time.