Hack the Plant

Hack the Plant, Episode 41: From the Archives

February 28, 2025 – For the final episode of the season, our host Bryson Bort reflects on four years and forty episodes of Hack the Plan[e]t, and picks a few favorites that are most relevant to the challenges ahead. 

This episode features excerpts from Hack the Plant; episode 8, DoD and Critical Infrastructure, featuring Lt. Col. Douglas Fletcher and Lt. Col Erica Mitchell; episode 10, The Congressman, The Commission and Our Critical Infrastructure, featuring former Congressman Mike Gallagher; episode 27, Managing Incident Responses to Critical Infrastructure Attacks, featuring Lesley Carhart; episode 28, Cyber Threat Intelligence Over the Past 25 Years, featuring Jason Healey; and episode 36, Supporting Ukrainian Electrical Grid Resilience in Wartime, featuring Joe Marshall.

Hack the Plant is brought to you by ICS Village and the Institute for Security and Technology. View transcript.

Transcript

Bryson Bort: I’m Bryson Bort and this is Hack the Plant. 

Today, we’re doing things a little differently. We’re closing out season 4 at Hack the Plant, and as we look ahead to what we want to accomplish in season 5, I found myself looking back on the work we’ve done so far. I wanted to share with you the 5 episodes that I feel are most relevant to the challenges we’re facing right now.

I started Hack the Plant in August 2020, driven by a deep conviction that we need to do something about the problem of critical infrastructure assurance: from education to action. Over 4 years and 40 episodes, we’ve hosted industry giants and practitioners, members of Congress and policymakers, and hackers and scientists; because with technology it’s the people, those on the front-lines that are making the difference. 

My first pick dates back to February 2022. Episode 8, DoD and Critical Infrastructure, featured Lt. Col. Douglas Fletcher, chief data scientist, and Lt. Col Erica Mitchell, and research lead for Jack Voltaic, a program of the Army Cyber Institute out of the United States Military Academy. Jack Voltaic was a series of exercises spearheading a major, multi-sector effort aimed at understanding critical infrastructure dependencies on force deployment. Congressional funding for the project has since run out, but the Army Cyber Institute is reportedly looking for ways to continue the program. I still continue to serve with this project on the West Point Cyber Science Advisory Board.

Every now and then, we get to see how small the world is… Erica was a cadet a couple years behind me at West Point where I leant her my car once. And the world can look pretty small for force projection too. The US Military is the most capable in the history of humankind, but it has to get there. I like to say, the best tank on the battlefield is the one you don’t have to fight. Our forces are heavily dependent on civilian critical infrastructure and Jack Voltaic was the first project to look at this with posts, bases, and ports across the country. I personally helped broker some of the agency relationships in my dual hat as a Board Advisor to Army Cyber Institute and a Senior Advisor for Critical Infrastructure to then CISA DIR, Chris Krebs.

Here’s Lt. Col Erica Mitchell on the project.

Erica Mitchell: So Jack Voltaic is a research series that has started to look at the interdependencies between civilian critical infrastructure and the DOD. And so the first iteration was born from a Cyber Mutual Assistance Workshop. So the energy sector for many, many years has had the concept of mutual assistance. If there’s a big storm in one part of the country, you’ll see teams being dispatched from all over the country to get those lines back in place and restore service in the effected areas. So a CW3 Judy Esquibel had the idea that what if we could do the same thing with cyber, and that was back in 2016. And so she put together a Cyber Mutual Assistance Workshop that brought in some leading industry players and worked on developing these public private partnerships and really bringing everybody to the table to say, what would it look like if we had to respond to cyber? How would our interdependencies affect that cyber response? How could we possibly assist each other?

And so after they had the first Cyber Mutual Assistance Workshop, they decided to try an exercise. So JV 1.0 was conducted in New York City and focused on a physical attack coupled with an attack on the financial industry as well as the subway. And what that showed was one, there are a lot of silos of excellence as we like to call them. Within each of the critical infrastructure sectors, there are communication pipelines where people report on incident, but there was not a lot of crosstalk between those different critical infrastructure sectors. And so after going through that, it actually led to the development of the New York City Cyber Command in order to have some type of unified cyber response and be able to minimize the amount of information that stayed within these silos. And we maintained a relationship with New York City and continue to do workshops with them even to the present day.

And after that, we looked at JV 2.0. We decided to take it a little bit further and do something a little bit bigger and looked at a hurricane coupled with an opportunistic cyber attack. And with that hurricane scenario, one thing we noticed was we kind of took the ports out of play. We wanted to know what would happen at our surface distribution and deployment command, which is an army… our TRANSCOM battalion that focuses on the movement of army equipment. They participated, but because the hurricane closed down the port, it kind of took a lot of the cyber off of the table for them.

And so we had a lot of good findings come out of Houston. Some of the similar findings from New York City that we still have a lot of these silos of excellence by moving to another large city, they’re very, very responsive to the physical, their reaction to the hurricane is on point, they know how to react to that. They’ve hosted several large events. They know how to react to any type of physical issue. But when it comes to cyber one, it’s hard to know that it’s cyber at first. The first instinct is, hey, I have a glitch in my system, let me restart it. Or hey, I’m experiencing problems, I wonder if anybody else is, let me just wait and see what’s going on. And so that is still an issue.

Bryson: Next up is episode 10, The Congressman, The Commission and Our Critical Infrastructure. We recorded this episode with former Wisconsin representative Mike Gallagher in April 2021 to learn more about the Cyberspace Solarium Commission. The Commission was a bipartisan, intragovernmental body whose goal was to help create a strategic approach to defending the United States from cyber attacks of significant consequence. The Commision was sunsetted in December 2021, but continues today as a non-profit led by Mark Montgomery and featured in episode 37.  

I was fascinated with the Cyberspace Solarium Commission from the first time I’d heard about it. Like many of us working in this space for years, it can feel disheartening looking at the overall system and the incremental progress that feels both too slow, and not enough. The Commission was a whole of nation effort to do something about it and enjoyed bipartisan support. Mike’s character really shines in this episode, he’s not just another politician, but someone who really cares about the problems facing our country. The latest, CSC 2.0, a nonprofit constituted out of it continues the work at https://cybersolarium.org/.

In this clip, Mike is answering my question about the role of government vs. private industry in cybersecurity efforts like the Commission. 

Mike Gallagher: Well, I think we have to recognize the fact that when it comes to cyber, 80% of the critical infrastructure is owned by the private sector. And that’s not going to change. And so that requires, first and foremost, I think, a paradigm shift for how the national security bureaucracy approaches the problem, and a recognition that in some ways they’re not the main effort, they are the supporting effort. And they culturally, they have to change from this posture of need to know to a duty to share, and add value to the private sector.

You don’t want the private sector constantly suspicious of working with the federal government, either because it’s going to compromise their internal information or hurt their bottom line. So, I think there’s an overall cultural shift that needs to occur. And I think the federal government then needs to distinguish itself in certain key areas where the private sector simply can’t compete or just isn’t involved in.

I mean, they are a very sensitive intelligence streams that the federal government has that it could do a better job practically sharing with the private sector, if their infrastructure has been compromised. There are specialized personnel that work in the private sector that can add value to the private sector. And then, I think if you go back to incentivizing the private sector to step up, we really want the culture of cybersecurity to permeate through our companies in the United States.

So the question is, how do you incentivize things like 1-10-60 reporting?

Bryson: What is 1-10-60? Breakout Time: It’s the measurement of the amount of time it takes an adversary to begin taking action on their objectives during an attack.  More specifically, Breakout Time calculates the time from the initial infection point of a given incident to when the adversary is able to successfully move laterally within the victim organization’s network, ultimately landing on the asset they are targeting during their campaign. Time to Detection — organizations should set a goal of allowing only one minute to detect an incident or intrusion (automated). Time to Investigation — the length of time it takes to find out if the incident is legitimate and determine next steps (containment, remediation, etc.). The best organizations can execute this process within 10 minutes. Time to Remediation — the period of time needed to eject the intruder and clean up your network, which may involve coordination with the business owner of that asset. The best organizations aim to perform these activities  within 60 minutes.

We’re jumping ahead to April 2023 now. Episode 27, Managing Incident Responses to Critical Infrastructure Attacks featured Lesley Carhart, Dragos Director of Incident Response for North America. Lesley and I took a deep dive into the inner workings of incident response and vulnerability assessment.

Most don’t realize it, but the majority of enterprise cybersecurity defense is built on detection and response. And how do you ensure breakout time is contained? Your IR isn’t your last resort, it’s the largest contributor to minimizing the inevitable breach through visibility and time to response. Lesley and I went on to speak together at RSAC in 2024 – prep is key for IR so we talked about the good, the bad, and the ugly of TTXs.

Lesley shared her experiences as a threat researcher in an ever-changing environment.  

Lesley Carhart: yeah, I mean, we see it all. We see it all in this field. It’s like working in an ER, kind of but, I mean, like, worse, like working in an ER in a mining town or something, you know, like we see, we see everything in OT incident response, and you know, that’s a mix of categories of incidents. We see insider cases, both intentional and unintentional. Insider cases we see, we see a lot of crime where so crime actors are getting smarter about where they’re doing things like ransomware attacks. They’re less haphazard. There’s probably less overall attacks now, but they’re more smartly performed. So they’re targeting more critical industries. They are targeting people who they think will have to pay less defended industries. So that leaves industrial kind of wide open for a lot of that type of criminal activity, financially motivated activity, and then there’s still adversaries out there who are adversary groups, who are more state style, who are building their capabilities to launch attacks in the future and conducting espionage, preparing to do sabotage. And that’s still happening, and they’re getting better at it, like there used to be, like this security through obscurity thing with ICS and there still kind of is like, if you want to do a specific thing to a process, like tamper with a water supply or turn off the power to a city, you know that takes a lot of knowledge of the industrial process and systems, because there’s a lot of controls in place, human and otherwise to prevent that from happening. But if you spend 10 years in a system, because you’ve got the funding to do as a is an army from a country or an intelligence agency, you’re going to know how to do that. You’re going to add the experts on staff who know how that system works, and you’re going to have built up the knowledge to be able to do the bad thing. And even those criminal activity actors, like the ransomware actors and things. They’re building up their knowledge base too, because they’re becoming, like, that’s a billion dollar industry now, like they’re very well resourced criminal groups, and if they want to do something bad to make money, they’re pivoting their operations away from, again, from haphazard ransoming, like they’re they’re going to be able to have those types of capabilities soon too, so that security through obscurity thing isn’t going to work much longer. 

Bryson: My next pick is the episode we released after Lesley! I sat down with Senior Research Scholar at Columbia University’s School for International and Public Affairs and cyber risk and conflict expert Jason Healy for episode 28, Cyber Threat Intelligence Over the Past 25 Years. Jason was here to discuss his October 2023 Lawfare article looking back at 25 years of White House cyber policies.

“Those who cannot remember the past are condemned to repeat it” For those of you who don’t know Jason, he has been one of the stalwart leaders of government cybersecurity and policy for decades. Cybersecurity is a young discipline, but with a dense history of success, failure, and rapid innovation. But, we tend to forget or not heed the lessons of the past in our relentless pursuit of improvement. Perhaps… a retrospective from someone who has driven a lot of it, studied more of it, and done the homework would help.

Jason explains what he gets into in his article. 

Jason Healy: Yeah, so the White House has been trying to get their arms around these solutions for 25 years. If you look back at the very earliest, White House document, presidential decision directive, 63 came out in 1998 and they don’t really mention operational technology, right? It’s very focused on it. They’re focused on critical infrastructure. 

But they don’t really make any differentiation about about it and OT and it’s, they’re so optimistic back then, Bryce, and it’s so cute to see they say, you know, within five years, we’re going to have, you know, most of the solved with 10 years, Americans, critical infrastructure will be secure, as if it was a one off, as if we could just get it right once, and then it would just be secure. But of course, we have intelligent adversaries, and we keep inventing new technology, so even if we could get it to a state of security, we would move off. 

So over the last 25 years, you’ve seen a lot of trends stay the same. White House doesn’t give themselves public deadlines like that anymore to have the whole thing secure. But they’ve continued to talk about things like information sharing. They’ve continued to talk about a lot of these themes where, you see the biggest difference, Bryson, is first off is in regulation. 

They started 25 years ago in saying, look the market. You know, our solution is going to be through the market. And we saw that all the way through the Obama reports strategies and the strategies from the Trump administration. The biggest difference in this one, at least in in the major content, is this new strategy coming out and saying that we need to regulate, that the market has failed, they talk about market failure at least five or six times and that we need regulation. And they push Regulation A couple different areas, from software liability or liability for software manufacturers to baseline regulation for critical infrastructure. There’s a lot more in here about operational technology. 

So it was nice to see these changes, because it’s like, if you’ve seen, you know, if you like sports, right? I’ve been watching a lot of American football lately, and you see teams that continue to lose you. Right? And you say, Okay, well, we shouldn’t be having consistency anymore between coaching, right? If you, if you’ve been at this for 25 years, your strategy shouldn’t keep covering the same things year after year, right? You’ve got to mix up your coaching style if you want to succeed. And we are suffering the same things from 25 years ago or even 50 years ago. So I am glad that the strategy made a break and started to go out in these new areas.

Bryson: This last episode is from season 4. Senior IoT Security Strategist at Cisco Talos Intelligence Group Joe Marshall joined us for episode 36, Supporting Ukrainian Electrical Grid Resilience in Wartime. When Russia invaded Ukraine in 2022, Joe helped coordinate a multinational, multi-company coalition of volunteers and experts to find a technological solution. 

Joe has the honor of being the first guest invited back on. HIs first episode, we looked at cybersecurity in Big Agriculture. A year later, he came to me and asked if I could solve a timing problem in the electrical grid. I didn’t have all of the details, but I certainly was stumped by the challenge. Well.. he wasn’t. And he doggedly pursued a solution that would help Ukraine maintain its grid for its people with the help of his company, Cisco. It is not hyperbole to note his actions saved innocent lives.

In this clip, Joe explains to me how he got involved with the project.

Joe Marshall: I was like, I was so enraptured by the stories of hardship, and innovation, and things they were doing to be able to just keep the lights on. And they mentioned something really offhand, like it was just like item one of 50 that they brought up to me and it’s actually the least sexy of all the things they were mentioning, had nothing to do with guns or missiles or whatever.

And they were like, yeah, we can’t even get accurate timing to work on our transmission grid because of jamming that is interrupting GPS communications. And this little thing in the back of my head went off and it goes, hey, that’s really important for synchro phaser management and for, you know, being able to monitor the health of the grid.

My time in the utility, had taught me that. But I haven’t thought about it in eight years. And I was like, oh yeah, that’s, that’s kind of important. And I kind of, I said to him, I said, hey, why don’t you guys just go buy atomic clocks, you know, that way you don’t have to worry about GPS timing. And the guy looked across me from the table.

He’s like, cool, are you going to cut me a check? Those are $30 to 40,000 a piece. And I need well over 40 of them. And I’m like, man, unless you’re asking for pesos, I can’t help you and even then, I don’t know if I could afford that. So it was a really interesting problem because I was like, how do you do grid synchronization and timing when you don’t have a disciplined clock that everybody steps to, or at least can measure and timestamp to?

And I was like, man, I don’t really know. But then I made a very naive assumption. I went, dude, I work for the largest hardware manufacturer in the world. I am stone cold, handsome and brilliant, I got this. I’ll figure something out. 

…….. 

And my little neurodivergent brain just wouldn’t let go of it. And it was just this weird one off thing. And I went, nah, we got to have something. The answer is we did not. And we would have to go make that thing, but the journey to get there was emotional highs and lots of lows, but we got it.

Bryson: That’s all folks! We’ll be back here with you on March 11th for season 5, where we’re tackling four of our most vital lifeline sectors – electricity, healthcare, food, and water. We know that our interconnectivity makes us vulnerable to our enemies – but what can we do about it?

Episode one of season 5 features my good friend Josh Corman, founder of I Am The Cavalry and Executive in Residence for Public Safety & Resilience at the Institute for Security and Technology. Josh leads the UnDisruptable27 initiative, a program of Craig Newmark’s Cyber Civil Defense initiative that works to drive more resilient lifeline critical infrastructure for our communities.

Josh was also one of Hack the Plant’s first guests. You might remember him from episode 2, Where Is The Cavalry? We discussed Josh’s idea for experts devoted to improving the security of medical devices, transportation, connected homes, and infrastructure. 

Look, the world is getting crazy. The good news is, we’ve been paying attention. The bad news is, it’s about to get a lot worse. But we’ll be ready. Together.