In this special episode of TechnologIST Talks centered on the implications of AI in cybersecurity, IST CEO Philip Reiner is joined by Heather Adkins, a 22-year Google veteran and founding member of the Google Security Team. As VP of Security Engineering, she built the global team responsible for maintaining the safety and security of Google’s networks, systems, and applications. Heather is also a member of IST Ransomware Task Force Steering Committee, where she provides support, guidance, and oversight on ongoing RTF lines of effort. Philip and Heather sat down to discuss the implications of AI on the offense-defense balance.

“There are no rules, and no amount of laws and regulations that we put in place are ever going to impact how they use it,” Heather said.

How did Heather get started at Google? What did the threat landscape look like in 2002, and how has it changed over the course of her tenure? Join us for this and more on this episode of TechnologIST Talks. See the transcript.

Simplecast | Spotify | Apple Podcasts | YouTube

Transcript

Welcome back to TechnologIST Talks! I’m Philip Reiner, CEO at IST and your host today.

We’re joined in this episode by Heather Adkins, a 22-year Google veteran and founding member of the Google Security Team. As VP of Security Engineering, she has built a global team responsible for maintaining the safety and security of Google’s networks, systems, and applications. She has an extensive background in practical security, and has worked to build and secure some of the world’s largest infrastructure.

And here at IST, as a member of our Ransomware Task Force Steering Committee, Heather provides support, guidance, and oversight on ongoing RTF lines of effort.

Heather and I sat down to discuss the implications of AI on the offense-defense balance.

“Number one, the way that we are using AI today and conceive of using AI today is very different than the way it’s actually going to get deployed. And this is true of every technology we’ve ever harnessed, whether it was fire or electricity.”

“What we are seeing in this realm is the fascination period. Both defenders and attackers are absolutely fascinated by the possibilities. And so you see an extraordinary amount of experimentation.”

We talk about how bad actors might weaponize AI in the cybersecurity realm–and what’s needed to stop them. 

“There are no rules, and no amount of laws and regulations that we put in place are ever going to impact how they use it.” 

“Automation is always going to increase the speed and efficacy of attacks. And then I also suspect that like defense, they’re experimenting with vulnerability discovery. Especially some of these stubborn weaknesses in software that I mentioned earlier. And this is where the rubber meets the road. Who is going to do that part more effectively?”

“Are we going to find vulnerabilities faster on the defense side or the offense side? And then how quickly can we fix them? And that, I think, ultimately is just to make a very complicated conversation, very simple. That’s the linchpin in this question. Who’s going to win is going to be who runs faster at the software vulnerability problem, in my view.”

And learn how Heather views areas for alignment between the U.S. government and the private sector.

“I think the most important thing is to create the table, to sit down at the table to talk through the issues.”

“We’re in a somewhat delicate situation with polarities and that we both want to move fast on AI because we know our competitors and national sense are doing so. And yet we want safety, right? So we want to both constrain and we want to make sure we’re still competitive. And that will require delicate balance.”

So how did Heather get started at Google? What did the threat landscape look like in 2002, and how has it changed over the course of her tenure? Join us for this and more on this episode of TechnologIST Talks. 

Philip: Hello, everybody. I am absolutely thrilled today to be joined by the one and only Heather Adkins. It’s really a genuine pleasure to have you here today. Looking forward to talking a little bit about AI and cybersecurity. So welcome to the podcast. Thanks for joining us.

Heather: Delighted to be here. My favorite topic, AI. It’s the hot topic everyone wants to talk about.

Philip: Cannot get away from it. I do think it is an interesting opportunity though, for us to be able to talk about this in real terms. There is so much noise in the system, there’s so much hype out there, there’s so much i think snake oil out there and so that’s one of the reasons why we’re really excited to be able to talk to you about this to maybe get to ground truth as to what’s going on out there. Let’s back up a little bit first though, and maybe just ask for the listeners out there to have you tell everybody a little bit about your journey at Google and how you have seen things, over the course of your career, develop and how the threat landscape has changed and all the rest of that.

Heather: Well, it’s been an incredible journey at Google. 22 and a half years, over 25 working on cybersecurity in some form. I don’t think we quite called it cybersecurity 25 years ago, but I think when I look at where we were 22 and a half years ago, let’s say, the internet was a really different place. It was still quite intimate.

There were big companies and we had just gone through the dot com bubble, but for the most part, it still felt very intimate and network operators still met each other on a regular basis. And the kinds of threats we were dealing with were, you know, a little bit of hacking, but you know, a lot of denial of service.

What is interesting to me though, is when I fast forward to where we are today, the kinds of things we are seeing aren’t really all that different. We’re still seeing exploits, vulnerabilities due to buffer overflows, cross site scripting bugs. You know, we’ve got these 25 very stubborn weaknesses in the infrastructure that we don’t know how to deal with.

What we are seeing, however, is a different kind of scale and a different kind of impact. We have more attacks than ever. They are more serious. They are conducted by more serious threat actors, but the weaknesses are mostly the same. And I think for me, that highlights a real problem, but also a real opportunity for us to, now that we’ve we’ve had a field now for a couple of decades to really go back and think deeply about how do we undermine some of these problems at the root, as opposed to some of the strategies we’ve taken over the last couple of decades, which is, you know, trying to add solutions and band aids on top of problems.

And that leads us to the conversation of AI, which I’m excited about.

Philip: We’ll get to the AI piece in terms of maybe how well positioned the ecosystem is for the step changes that we’re seeing now. You’re talking a little bit about how the vulnerabilities, the gaps have remained the same, but the scale of everything else has really begun to alter. Talk a little bit about from your perspective, obviously with Google at the forefront of this, how have companies changed over the course of your tenure in doing this, have you seen them really begin to take this more seriously?

Is this much more of a board issue than it once was, or is that still nascent from your perspective as well?

Heather: We’ve gone through several phases of realization. About every five years, we have had some kind of major breach. You know, I think about, you know, Google in 2010, we had a very notable breach. RSA followed the year following. And since then, we’ve had kind of every year, there’s a new sort of awakening.

Watershed moment is usually the headline you see in the news. What I’m actually a little worried about at the moment is that there is so much awareness now that it’s almost normal. I talked to teams outside of Google about, “did you hear the latest news today about this company and this breach? Did you hear that, you know, this person got ransomwared?” and nobody’s ever heard of it.

And those are the kinds of moments which would have been the biggest moment of the year had they happened a decade ago. So I think we’re somewhere in between enough awareness and too much. It’s kind of overload and just normalized now. But I do think that the part of the ecosystem that is aware now that’s helpful is on the government side and the role that government might play in various jurisdictions around the world to help normalize different conversations.

Now, I’ve been very, very lucky at Google that our leadership has been engaged in cybersecurity from the beginning. If anything, they push us faster than we want to go sometimes, but I know not every company is in that position and, you know, a lot of companies are dealing with very challenging competitive environments.

They’re balancing the velocity of their business with innovation and kind of speed of dealing with technical debt. This is a challenge for every business on the planet, different in hospitals than in banking, but I think everybody is naturally constrained. That’s kind of the nature of business. So I think companies are more aware.

I think we’re certainly in a better place than we were before, but we are still very challenged by the solution set to fit into that unique problem space that business has where they are constrained naturally just by resourcing.

Philip: Okay. Something you said there really caught my attention. I’ll dwell on it just for a second, then I promise the listener we’ll get to the AI stuff. That government has increasingly been able to help normalize some of these conversations. What are some examples of that from your perspective?

Heather: I love the work that CISA has done, well, since the founding of CISA, first under Chris Krebs, now under Director Easterly. To normalize the conversations at an executive level, I remember going into a number of conversations a decade ago where, you know, an executive would say, well, that’s a really interesting theoretical issue, but it would never happen.

But to now have very senior people in the government say, not only is this happening, it’s happening on a scale that is very challenging for us, and it is very likely to happen to you. And it is a good way to reach an executive audience who is otherwise focused on other parts of the business, so that’s been helpful, and I think they’ve also done a good job in bringing industry to the table to have some of those conversations and then, you know, especially under the last couple of years ago at CISA, we’ve seen initiatives like secure by design come out to start to set the tone around. What does it mean to make a good product? And secure by default to sort of operate a good product as a business.

And I think all of these things have been just extraordinarily helpful in helping the practitioner on the ground be able to make the case to management that we care about cybersecurity now is the time where we need to elevate and prioritize a few things.

Philip: I think it’s having just been on an international trip this past week, where the government is not necessarily nearly as involved, and it doesn’t help bring those execs along. It’s really a fascinating point. I will also for the listeners out there do a little bit of a plug for something you just wrote the other day about secure by design and some of the work that Google has been doing on that front.

I think it’s actually world class global market-leading work that you’re doing there. So, kudos on that front. Maybe we can pivot a little bit to the AI piece, right? So from IST, we just released this work that we put together. It was a large endeavor where we went to a lot of folks across the ecosystem to try to get to ground truth, right? So instead of just reading headlines, we went to a lot of the operators to ask them what they’re actually seeing in terms of the offense defense balance, in terms of how they see AI being deployed inside companies for the defender, and how they see the malicious actors out there taking advantage of this stuff. Very curious just to kick off the conversation here. How are you seeing it? Have you seen AI really change much, or is it still that hype that’s out there? How would you speak to it just kicking it off?

Heather: Well, let me start at a really high level, not in the fiber conversation at all. But I think we have to realize that first off, AI is not new. It has been something that scientists have been deploying and working on for quite some time. What we have in the last two years is really successful deployment of large language models.

From Google’s vantage point, we’ve been watching developments very closely. Most of this work, the early development work, happened at Google. It starts with a Transformer paper in 2017 and it sort of exploded from there. And when I think a little bit about how just generally, people are looking at large language models at the moment, it’s mostly from a vantage point of fascination. I’m inside an AI company, so of course it’s all we talk about. But if you were to talk to an average person on the street who is not in an AI company, the way they are experiencing this revolution is very different. And I think that’s incredibly important to understand that in technology, we are very much in a bubble.

If you are impacted by AI at the moment, at least here in the United States, it’s probably because you’ve got a student and you’re worried about plagiarism or you’re a teacher and you’re worried about plagiarism, or you’ve heard about deepfakes or your kids are deepfaking online. It’s not as if billions of people are sitting down and having a conversation with a chatbot every day.

That’s just not the reality of what’s going on. So I think that’s an important place to start this conversation is to realize, just like with any technology revolution, the technologists are the ones who are deep inside the bubble. And we are sort of very deeply thinking this is going to change the whole world, but that doesn’t mean the rest of the world cares.

Okay, so I think just that with that reality in mind, I’ll kind of make two points. Number one, the way that we are using AI today and conceive of using AI today is very different than the way it’s actually going to get deployed. And this is true of every technology we’ve ever harnessed, whether it was fire or electricity.

You know, when we first made fire to cook food, I doubt we ever dreamed we’d invent a steam locomotive. Just keep that in mind, and then the second point I’ll make before we kind of dive into the cyber piece is that AI in its most useful form is not a chatbot. It is integrated into workflows. And my best analogy here is like calculators.

We used to do computing of numbers on paper. And then suddenly the calculator came along and everybody’s like, I’ve got this other device on my desk. It does math. I know how to do that. And it took a little while before calculators became kind of integrated. And now, you know, you’ve got a calculator on your watch. Every computer you own, your car has a calculator in it. And it became so integrated into technology that you just, you don’t really think of it there, right? Like, you know, we’re recording this podcast. There’s a calculating engine behind the technology here, right? So it’s going to be the same with AI.

And when you go to the doctor, you’re not going to know that your medical assistant used AI to help write the notes and the radiologist used AI to help interpret the image. And in your life, there will be lots and lots of little invisible AI-assisted agents behind the scenes, just helping make everything a little bit better.

And that is the way most people will experience AI, just realistically, but shifting to cyber. So those two things in mind, shifting to cyber, what we are seeing in this realm is the fascination period. Both defenders and attackers are absolutely fascinated by the possibilities. And so you see an extraordinary amount of experimentation on the defense side.

Actually, let me start with the attacker side. The attacker side is completely unrestrained. There are no rules, and no amount of laws and regulations that we put in place are ever going to impact how they use it. We’re seeing, of course, the use of deepfakes to make social engineering phases of attacks much more effective and synthetic content around phishing emails, collating information together about victims that make that more compelling, speed.

Automation is always going to increase the speed and efficacy of attacks. And then I also suspect that like defense, they’re experimenting with vulnerability discovery. Especially some of these stubborn weaknesses in software that I mentioned earlier. And this is where the rubber meets the road. Who is going to do that part more effectively?

Are we going to find vulnerabilities faster on the defense side or the offense side? And then how quickly can we fix them? And that, I think, ultimately is just to make a very complicated conversation very simple, that’s the linchpin in this question. Who’s going to win is going to be who runs faster at the software vulnerability problem, in my view.

Philip: Let’s unpack that a little bit. So we had a conversation with a CISO recently who said that he feels that within the next year vulnerability discovery is probably going to increase tenfold. And the immediate thought that struck me was there’s no world in which people can keep up with that sort of a battle rhythm. What are you seeing today, or just, you know, in the recent past, maybe what’s coming soonest as to the ability for folks not only to find those vulnerabilities, but to be able to respond. It’s not like we’ve got a whole bunch of extra people, right, to throw at these newly found vulnerabilities to start cleaning them up.

Heather: But this is not a new revelation. We already know teams are having difficulty patching. Today and have for the last five to ten years to keep up with that cadence. It’s one of the reasons CISA came out with the known exploited vulnerabilities list, the KEV, is to give government agencies at least a little bit of a hint about where to prioritize.

If you can’t do everything, at least prioritize these that we know are being exploited. I think we’re going to have a difficult couple years, but I think what we are driving towards is to change the ecosystems, expectations, and assumptions. We have to get away from this notion that, as an enterprise administrator, you are going to set aside a large portion of your workforce to just patch and to qualify patches.

We are absolutely going to have to get comfortable somehow. And again, I will not pretend to understand every enterprise’s challenges, but we will have to get comfortable with more frequent and automatic updates, and there’s just going to be no way around it. We cannot slow down attacker discovery. I think the other thing that is going to emphasize some of the work that CISA is doing on secure by design around classes of vulnerability.

There is a natural question of if there are so many vulnerabilities in the software, how do we just prevent new ones from happening? And it turns out, now, I’ve seen a few studies on this. It turns out most of the vulnerabilities that get a CVE and get released and patched are relatively recent. It’s not as though these have been in the code base for 10, 15 years.

There are a few, but the large majority of them are being introduced by today’s developers and that seems like a preventable problem. We can move more and more of that discovery into the kind of design and, you know, before it’s released can we find those bugs first?

Philip: It was interesting, and if I’m not mistaken, I saw something to the effect yesterday that your CEO had made a comment to the effect that 25 percent of new code is being written by AI.

Heather: Yeah, and I think that’s low hanging fruit. It’s things like autocomplete. It’s not novel new code as in “please write me a linked list” or “please write me a bubble sort,” but it’s sort of anticipating what the developer’s writing and sort of auto complete of code, right? So you still have the kind of human element in that.

But there will come a time when we have the ability to help the developer use secure frameworks in an automated way, right? We’ve been very successful reducing cross site scripting errors at Google this way. This is to say, you know, instead of inventing your own library, which may be flawed, or not even using a library, but just writing your own function, here’s one that’s already safe by default to do this very complicated thing like cryptography or accepting sanitized input. You could also imagine as you’re writing, like maybe you’re in a code base C or C++, and you accidentally create a buffer overflow. We should be able to highlight that for you, either as you’re writing or in a testing harness, like a fuzzing framework to catch crashes and to have you take a look at them.

And this all sounds very difficult, and the people who do this work know how challenging it is. But imagine we extrapolate five years of research out from here. These things are possible. And the more you can reduce the amount of vulnerability you create, the less opportunity do threat actors have, and the fewer patches we then have to push later on.

And so I see, like I said, we’re going to have a really difficult couple of years, I think. But we’re going to drive towards that. Find the vulnerability early and reduce the amount of patching we have to do. But then we also have to get the ecosystem comfortable with automatic deployment, automatic patching.

And I realized this year we had a big CrowdStrike outage and everybody’s very nervous about automatic updates, but there are safe ways to do that. Automatic rollback. There are strategies for mitigating all that risk. But ultimately, you know, just to, you know, and I tell the teams at Google this, by 2050, and that’s a long way out, but by 2050, I do think we will have systems that are relatively self healing.

Philip: You think it’ll take that long?

Heather: I think it’ll take us less time to build the technology. I think it’ll take time to get comfortable on the business process side, on the regulatory audit side, and just on the practitioner side on the safety elements of that. We’ll see kind of small deployments of it earlier than that. But you can imagine a system that’s, I’m under attack, looks like a vulnerability.

I’m going to detect that vulnerability, patch that vulnerability in real time, or get an update, like my vendor’s just released the update. I’m going to pull that in. hot pad in it. Those kinds of things will become more prevalent and normalized, standardized by then.

Philip: So there’s this pull to ask you about agentized capabilities in that regard. But I want to back up real quick. In terms of what you’re seeing, in terms of what adversaries are being able to do on this front. We talk a lot about this offense-defense balance, Google and the team there, Phil and Charlie, I think put out a piece not too long back about how AI can really tilt things to the defender’s advantage and actually flip that dilemma. What are we seeing? What are you seeing in terms of how AI capabilities are actually being used? In real time being put to use by the bad guys that are actually novel ways that you guys are kind of like, “whoa, okay, wait a minute, they figured out a different way to do things because of these tools,” or they’re not really there yet.

Heather: I haven’t seen anything novel. And I think largely because the non-novel things work really well. Phishing, using deep fakes to social engineer the help desk. Now it’s very probable that novel attacks are being developed that maybe we necessarily don’t see. Remember, there are lots of large language models that are not developed in the West, just to put a little bit of a euphemism on it. And we may not have visibility into them for quite some time. So I wouldn’t be surprised if they’re there. We’re not seeing it. We’re seeing everything we would expect to see in a fascinated attacker environment. Everything the defense is trying, attack is trying as well.

Philip: In terms of what you had expected to be happening by this point, I’m curious, this was a question we asked a lot of people, right? What did you expect to be happening by now that isn’t? What comes to mind on that front?

Heather: I think it’s playing out exactly as I would expect, to be honest. I do wonder if we have good enough clarity on vulnerability discovery. The rate of zero days increasing would suggest there’s some sort of automation happening behind the scenes. Whether that’s large language model-assisted or not, I don’t know.

But otherwise, it’s obvious that you would try deep fakes and leveraging translation services, bringing together large data sets from data breaches to sort of correlate victim profiles. And we’re certainly seeing a lot of creativity used with that data, but I’ve not, I’ve not been surprised by anything.

I think if you’ve been around in the field enough, you’ve seen enough to know. “Oh yeah, of course you could do that.”

Philip: So to date, this was another piece of the investigation that we did in the report that we just wrote about how the security operations team is actually going to be able to take advantage of these new tools, how it streamlines and actually enhances those operations. Something you said at the very beginning I think is really important, which is. We don’t really know for sure how these AI tools are going to play out in reality. And I think a lot of our insights in terms of the investigation we did, that really bore true. It was in a lot of very simple ways that it’s actually streamlining efforts. I’d be curious if you could speak to that a little bit for folks out there to just let them know what the, kind of the cutting edge looks like.

Heather: Well, first and foremost, the low hanging fruit in security operations is data summarization. This comes up in a number of places. Let’s say, for example, you’re in a SOC. We’ve spent a long time as an industry trying to bring together different data sets so we can do detection better. What that means is that the analyst has a data overload.

If you’re doing correlation across 30 pieces of enterprise data correlated with an alert, you have to do a lot of reading for every alert. And so, if you can sort of land in an alert and it’s automatically summarized that Alice went to this website and it looks like it downloaded malware, and that malware is reaching out, like the SOC analyst is going to have an easier time diving into the data with a high level hypothesis.

And then similarly on the other end is writing the report of that alert to say, “actually, I talked to Alice and this is not a big deal.” And then for that analysis to be later available for similarity matching to other events or down the road, if it turns out, actually, the analysis was wrong. Those are the parts of the job that A, nobody likes doing and B, you’re difficult to get right and to sort of interpret later on. 

The other place that I’m actually really delighted is with malware analysis. And VirusTotal is a part of the Google family, and they’ve spent some time trying to figure out can Gemini, which is our large language model, help malware analysts kind of more quickly get through either the reverse engineering task or summarizing scripts, things that may take a week. Or, you know, even longer for someone to do if they’re early career or whatever, and actually just automatically do it for you.

And this will make not only things faster for people doing analysis work, but also organizations that don’t have experts now would have access to a capability that they wouldn’t have had access to before. So again, summarization, anything that’s language based, that’s easy to summarize, it’s going to be low hanging fruit and you’re seeing that kind of get deployed into commercial products.

Google is also investing in that in our Google SecOps product suite. And then, of course, for ourselves as well internally. I think on the operation side, you kind of look at everything a security operations team has to do, whether it’s building internal help desk tickets, which nobody likes doing, answering policy questions, doing configuration reviews, configuration checks, configuration generation, there’s probably a hundred or a thousand places where you would just very quietly put an assistant in place to just accelerate and make the job easier. And imagine what that will impact on talent retention, training, and the cybersecurity workforce. There’s a whole range of opportunity here that we are starting to see very early signs of.

Philip: And so one of the things that we heard from some folks, and I wanted to ask you about within that is, as you move in that direction with those assistants and as those assistants get better and better, you know, this as well as anybody, right? This automation bias, this becoming very trustworthy of these new capabilities, because they do take some of that low hanging fruit off of our plates. What sorts of vulnerabilities or risks have that introduced that you’ve seen today? How can folks be thinking about how to mitigate that and anticipate it best to make sure that they’re, you know, by introducing these new ways to make, create efficiencies and help out the workforce, how do they avoid any sort of downsides at the same time?

Heather: Really interesting question, because there is an instinct here that, gosh, the machine might get it wrong. But I never hear anybody saying, my analysts are wrong 50 percent of the time. Believe it or not, your analysts were wrong some percentage of the time. There was some great work actually done by the self-driving team at Google on this, and they published extensively on it because they have an artificial intelligence model behind the self-driving car, of course, and they have a high percentage of accuracy of the machine making the right decision at the right time, but it’s really difficult for people to accept that a car would drive better than they do. Of course, we’re all amazing drivers, right? So my first piece of advice would be to try to make this a data-driven exercise and really, well, first and foremost, keep a human on the loop in the beginning. Let this be an assistant to you, not making independent decisions.

And I think we’re going to be in that stage for, for quite some time. But secondly, also doing evaluations to say, the human made the decision with this assistant, how accurate was the human versus the assistant and try to make a, convince yourself over time that, you know, as the technology gets better, that you can rely on it more in the same way you rely on your humans who make mistakes every single day.

Philip: You test them out though, yeah.

Heather: Again, remember, this is assistant technology. We’re not thinking about replacing humans, especially where it really matters. Although I would love an AI assistant to prove all the expense reports for 10 dollars of course, right? Like maybe there are some places where I’m willing to take a little bit of risk, right?

But we’re going to have to create those risk models for ourselves. And that will take time, those frameworks. So number one, keep a human on the loop. Number two, make a data driven decision and keep abreast of the new mechanisms here, because I think we will end up finding new techniques. So I can imagine there might, I imagine at some point, some venture capitalists will put some money into, you know, what does it look like when you take two large language models that are different to do the same task?

And then they will sort of have to negotiate which one of them is right. Like I’m sure somebody will try that or somebody will try creating a large language model auditor who comes in after the fact and samples things and tells you whether it’s right. So keep an eye on technology advancements there because we will invent ways to validate our assumptions and to validate the accuracy over time.

Philip: So let’s come back to this other thing that I think a lot of folks have been seized with, and this gets to the assistant idea a bit, this idea of agentic AI and where it’s taking us. I have spoken with folks who are using AI agents in the cyber domain inside their organizations to help improve workflows. What is agentic AI? What does it mean to the Google security team? How have you guys been thinking about how it’s going to work for you and how have you been deploying it?

Heather: I try to draw this back into kind of the old software model where you might modularize functionality. And so if you were building a SEM, for example, an event manager, you’d have a module that could query VirusTotal and with a hash and pull back something. You’re essentially just swapping that module for an intelligent module, an agent.

And I think that’s a very natural way to extend the use of AI into many parts of existing, established security operations. I think the imagination and fascination part of the technology revolution we’re undergoing leads people to sort of take that scenario and go as far with it as you possibly could.

So gosh, what if my agent could not only query VirusTotal, but actually go down meta download Metasploit and exploit a vulnerability against another system and you sort of can go on and on and on and then get very frightened by it. So what I would say is we should think about agents as having very confined roles. You want to be able to define the role because you want to be able to measure the performance of how they’re doing it, just like you would any employee.

And I think the natural state we will end up with is confining the system in a useful way and coming up with frameworks to think about that. Taking us out of the cyber realm, for example, if you are a radiologist working with x-rays, you would want an assistant to help you interpret what you’re seeing, but you wouldn’t want it to go on Facebook and  chat with your best friend, right?

You can find the agent to doing exactly the role that it’s supposed to be doing. And I think that we will see that become more of a norm and a standard over time because we will want, as much as possible, to be able to reason about the system, even if the system is sort of exhibiting these hyper intelligent decision making kind of evaluations on our behalf.

Philip: Very, very quickly. And I know it’s, I think fair to say that this is more of a hypothetical and a theoretical, how does an adversary begin to think about agents and what they can do on their behalf?

Heather: Well, that is an interesting thing. Question because, of course, they’re not going to have the same constraints. At the same time, they will have many of the same threats. So, for example, you’ve got a really intelligent agent that can hack the planet and do whatever it is that you may want to do, but you may suddenly find your agents not doing it on your behalf, or that it’s ratted you out to the police.

Or, you know, introduced a really bad OPSEC decision. So I think there will be a lot of experimentation, but we saw it on the internet in the early days with the wormification of bugs. So you had like SQL slammer, you had conficker, things that get out of control and then suddenly they’re outside your control as the attacker.

So you don’t get to use them. Like you don’t get to leverage them because it’s just so vast. I think the best proxy for this, there’s, there’s some great work out of CSET on AI military thinking in China that actually the kind of gamification piece of this is largely unknown.

If you were to put out autonomous weapon systems, whether they’re cyber systems or kinetic, you do lose a little bit of battlefield control command and control. So I think there’ll be a lot of experimentation. I think the internet might be a little bit messy for a while. Like I said, I think it’s going to be a difficult couple of years, but if you’re a serious threat actor and you have serious goals you’re going to probably think twice about just what you want your agents to do on your behalf, because it may not have the context. Your business context as a nation state actor is to not get caught. And there are parameters upon which you operate that the AI just may not understand. It may not have that context and be able to enforce those parameters for you.

So I think for serious stuff, it might take a while for people to figure out how to use it.

Philip: I think we’re coming up on time. We’ve talked quite a bit about the offense-defense balance, we talked about how it’s going to be a messy couple of years, we’ve talked a little bit about kind of where things are going. I guess the one last thing I’d ask them to kind of leave the audience with, if you will, how can the public and private actor, you spoke a bit about how gov has gotten better. How can public and private actors be thinking about how to work more closely together? I know you guys have the CosAI work that’s going on that’s very collaborative, private sector, everybody working together to come up with open source stuff. How can public and private come together on this and work to tilt that advantage toward the defender?

Heather: I think the coalitions are incredible and I am so pleased at how quickly they spun up. I’ve been impressed at how quickly government has engaged. They didn’t start regulating cars for like 30 years, so I’m happy to see them jumping in on the AI piece very fast. So I think the most important thing is to create the table, to sit down at the table to talk through the issues.

We’re in a somewhat delicate situation with polarities and that we both want to move fast on AI because we know our competitors and national sense are doing so. And yet, we want safety, right? So we want to both constrain and we want to make sure we’re still competitive. And that will require delicate balance.

And then also there’s just a set of societal norms that are not set for this new technology. You know, what we’re here in the United States, what are the American people comfortable with? We have not yet come to that conclusion. I suspect we will. The EU is going to have a different view. China is going to have a different view, India, a different view.

And the only way to work through these things is to have the tables to sit down and work through it. So I’m excited about things like the Frontier Model [Forum], which is bringing together all the Frontier Model people, and they’re having conversations about how do we just build safe tech. And then we have things like the Coalition for Secure AI, COSAI, which is looking at how do we give enterprise businesses the playbooks they need to adopt AI safely.

I’m excited about that because it will help business rationalize some of these frameworks they’re going to need. And also it gives us a place to bring the problems back and to discuss them as they emerge. Like I said, this is going to play out very differently than we think it will. And we’re going to get surprised.

Those of us in cyber get surprised, you know, once every year or so anyway, but you know, we’re going to be surprised at how this stuff gets used and we’re going to need to sit down and work through it as a community. And so I’m very excited about the coalitions and all the collaboration. I have a direct line to the folks at OpenAI and Anthropic and Meta, et cetera.

It’s a tight community. So I’m pretty excited about that too.

Philip: Well, this has been a fantastic conversation, Heather. Thank you so much. I know you’re incredibly busy. So thanks for taking time out. I do think you’re incredibly spot on. We don’t know how it’s going to play out. It always manifests in ways that we can’t really even anticipate, but through that sort of collaborative activity, we can at least try to anticipate as best we can. Thanks for all you do, thanks for being with us here on the podcast.

Heather: Thank you for having me.