Swapnil Bhartiya: When it comes to AI governance security. In most cases, it may seem that your teams have AI governance under full control, but the data says otherwise. 93% of companies already had an AI related infrastructure incident this year, even as they felt confident in their safeguards. And that confidence gap is exactly what Spacelift’s New Infrastructure Automation Report exposes at a time when agent or agent AI starts to write infrastructure code on its own without any human supervision. And today we have with us once again, Jonah Kowall, SVP of Product and Design at Spacelift, to explain what that gap is all about and how that can be tested or how that can be bridged. First of all, Jonah, it’s great to have you back on the show.

Jonah Kowall: Good to be here. Thank you for having me back on.

Swapnil Bhartiya: Yes, my pleasure. Also, because this is a very, very important discussion. As AI is moving into production, agentic AI is starting to write code without less and less human, you know, you can say supervision. Now, when it comes to organizations, most organizations, they seem very confident about their AI governance, their practices. But if you look at some of the data, that tells a totally different story. Can you talk about why is there this gap between how safe teams feel and how prepared they actually are when it really comes to serious blind spots in enterprise IT Now, AI also kind of makes things more challenging because they do a lot of things on their own. They’re also good at hiding things. They’re also good at hallucination. So the problem is not as easy as you just look at configuration. And you do know the blindest parts. So talk about it, please.

Jonah Kowall: Yeah, I think there’s a few interesting facts on this that the survey brought forward. I would say that many organizations actually have governance tools. The number there is 38%, but most of them have not turned it on. So that is an interesting anecdote. And, and the second one is that 86% of organizations are confident that they’re covered, but only 30% of them have even written a policy down, which to me is kind of the first step is you write something down, then you have a discussion, then you think about how you implement that. You go through a pretty normal curve. So there’s definitely a lot of concerns here where organizations are just not doing the foundational work before they can even think about having that governance.

Swapnil Bhartiya: And the confidence, what is the reason for that? Because we have been talking about AI for a long time. Whether it’s CEO, cto, no matter who talks, who you talk to, they will all be bullish about AI. The adoption of AI, all of their products will have AI label there. So you would assume that they have also followed those so called best practices of right governance, right policies in place. That is even more challenging when it comes to regulated or industries that have to be compliant. So why, what is the main cause? Where you see the gap between believe that we have governance place versus actually governance place, Is it about awareness or they actually, what they do is they just deploy an LLM and they assume that LLM will deploy all those policies in place. What is actually going on?

Jonah Kowall: Yeah, I mean I think it’s interesting that 93% of folks have had incidents, but they don’t really consider those a governance failure. They consider them like a misconfiguration. So I think that due to the speed of AI, things are slipping through the cracks which is why the governance is key. And as I mentioned before, the first step is really writing it down. Then you can think about how you actually implement the guardrails, provide the governance that you need to improve there.

Swapnil Bhartiya: Really can you also talk about, I mean as you rightly mentioned, that there’s so much code is coming out, AI is getting everywhere. Is it because they are not prepared, they fully don’t understand how deep AI can go into the system or is it because the AI driven velocity is just too much, is too overwhelming for teams? Systems responsible for deploying and governance code can’t keep up. What is the actual cause?

Jonah Kowall: So in the survey we found that 89% agree with you, which isn’t surprising. Velocity is up, but also infrastructure teams workload is up, that’s 86% so pretty much the same. And that’s also what I hear from clients pretty consistently. And so this means that security vulnerabilities are being discovered faster, they need to be patched faster because now generating a zero day vulnerability is pretty trivial compared to before. So there’s a lot of strain being put on pipelines. Just I don’t know when this is going to air, but just yesterday there was a really nasty six hour outage on GitHub and we see these pretty regularly. So I think it’s not just a strain on the systems but also the systems that we rely on like GitHub, almost every organization. So there’s a lot of like downstream effects that are occurring here. And so that’s, that’s kind of the part of the challenge. Other challenges are that many vendors don’t really have the governance controls in place to be able to manage these things. So there’s a lot of issues. But I would say that velocity is a big cause, but it’s not the only reason why governance is not there. But it should make people think, hey, we need to get our governance in order. Because I don’t think it’s slowing down. I think it’s just going to speed up. And as we get agents and teams of agents, this compounding effect isn’t going to slow down. It’s not going to all of a sudden, like level off anytime soon. So you got to do the work and you got to start with writing that policy and thinking about how you put guardrails in place for the future.

Swapnil Bhartiya: How would you differentiate or separate leaders who understand it? And not only understand it, they also use right practices. Right practices, kind of overused word. But yes, they have right practice in place. They have right governance, right policies in place. They have right. Also, sometime when it comes to AI, we all rush toward the biggest model. But that is not the truth. Sometimes the smallest LLM will do a much better job than the biggest one. So can you talk about what actually separates leaders from the ones who get exposed and compromised?

Jonah Kowall: Yeah, definitely. So we sort of separated these groups into different tiers of maturity. And that was sort of based on the way that they answered and how, I guess, ahead of the curve they are in terms of having this stuff in place, both in terms of governance and automation. And there’s a pretty nice spread between what we call the pioneering organizations, the ones that are ahead of the curve, or the exposed ones, the ones that are really behind the curve. But there’s some interesting kind of proof points is that when you go to the high end, 71% of those pioneering organizations have policy being enforced in their pipelines. They also have most of their infrastructure, 75% covered by infrastructure as code. 80 to 92% of their infrastructure is automated. So you can see they built a lot of this tooling. And when we look at the exposed, the other, the opposite end of the spectrum, only 4% of them have their policy enforced and their IAC coverage is 15%. So a very small percentage of their environments. So it’s interesting to see that these best practices and platform engineering and automation are clearly tied to, you know, organizations that make time or effort into solving those problems. So it’s quite interesting to see the spread. And maturity isn’t something that you buy with headcount. It’s not something that you build with vendors only. It’s really before you, whether you’ve spent the time before AI has accelerated or not essentially. But that doesn’t mean that if you’re behind the curve that you shouldn’t start doing some of this stuff upfront. Whether it’s automation thinking about infrastructure as code, better code review, which is part of IAC implementing the GitOps kind of workflows to ensure that things going into production are properly reviewed and understood. There’s a lot of best practices that folks can start doing whether they’re using open source or commercial tools like this stuff’s definitely important and will save pain down the line.

Swapnil Bhartiya: Now let’s talk about wipe coding. First of all, it has enabled a lot of non technical people to be able to create software just the way iPhones have. Or a lot of tools have enabled non professional photographers to take great pictures. The problem happens when you write a code using wipe coding and just push it in production without doing all the due diligence, without hardening it. And that’s when even when the open source developers, they complain about the code submission because you just wrote. It’s more like somebody asked you to write a blog post, you just wrote something what came to your mind and send it to the editor for publication. That is not how it works. Same thing should be the case with wipe coding. Now what happens when AI generated infrastructure code moves into production with little or no human review it?

Jonah Kowall: Yeah, this one’s kind of interesting because in I have a few reflections on the survey itself, but then also something that we’re thinking about solving with AI in our product that I’ll just mention because it’s kind of an interesting take on it. So 76% of the survey respondents would approve an AI generated infrastructure code changes with little or no review. I’m not that surprised by that. Well, I guess I kind of am surprised by that. But I think there might have been a second question in there, which is what level of change are you comfortable with that AI generates that are relatively safe? I think that that is where the interesting question is because certain things that AI generates are relatively innocuous. So for example, if I’m in a development environment, is it okay to not review infrastructure changes? Maybe because if it breaks, you just revert it and fix it. You know, when you’re talking about production, which isn’t part of the question, that’s a different situation and definitely requires the review. 33% of respondents would do no code review at all. And so it’s interesting to see the range here. And a lot of people have actually like already done this in their environments. It’s just not clear what kind of environment they’ve done that in. Based on the survey. So hopefully we’ll tighten that up. And of course we, you know, if you do have those controls, like I mentioned, in development environments in certain situations, I think it’s okay to do this. But obviously you do want to make sure you’re doing the testing and you want to make sure that you’re doing some code review or something of the sort before things go into production. And so I think the organizations that do have the governance and controls can make those decisions better than the ones that don’t in terms of what’s safe and what’s not safe. The other piece here is the automation and testing. If you create a really robust test suite that’s part of your pipeline, where you’re actually exercising and testing everything and ensuring that the quality is there, code review becomes much faster because you know that it’s going to work a certain way. So I think there’s a lot of different caveats in here around vibe coding that can make it okay as long as the testing is good. But obviously human review, manual testing is definitely important for those critical changes, the potential challenging changes that you may have.

Swapnil Bhartiya: As developers start using more and more AI to write code as their wipe coding. And the biggest problem is they wipe code, they push all the code in production and it has. So what I’m saying is that just the way we had practices like shift left, you know, the whole DevOps DevSecOps, we should have some practices in place so that they can freely use AI to write code but follow those processes, follow those practices. So when the code is pushed in production, it is as clean as if it was written by humans. Though actually human written code is not that clean, otherwise we won’t have bugs.

Jonah Kowall: True. I mean, I think although AI is improving in quality, it still hallucinates, it still has issues, it’s always good to review it. It’s interesting that a lot of these new models that come out, they often go on what people call side quests. They just start like working on something unrelated to what you asked it to work on, or they just find something when they’re investigating something else and then they try to solve that. And it’s really hard to keep them focused, especially when the models change and the behavior changes of these models. So I think that those challenges are always going to be there. You’re always going to need to have some level of human review, especially in these critical systems. Now if you’re just building an app for fun or that helps you do scheduling or something that’s not mission critical, that’s fine. But if you’re coding software that is really required for work, it can be really disruptive if vibe-coded software is just released without review. I think it is good to have professionals in the loop and people that understand the code. And just like anything we generate with an LLM, we still own it, even if the LLM generated it. I can’t blame the LLM, it’s mine. I own this, I review it, I open the pull request and it’s in my name and I should know what’s in there just like anything we generate, whether it’s an email or a presentation.

Swapnil Bhartiya: Looking at the survey once again, 89% of respondents plan to adopt agentic AI for infrastructure operations, while only 30% plan to adopt a governance policy. That is a recipe for disaster. Can you talk about what actions teams should be taking right now? We have been talking about all these practices, but this gap should not be there. You should actually start talking about governance before adoption. So talk about what actions teams or leaders should take right now.

Jonah Kowall: Yeah, I mean, obviously shoring up governance is an obvious one. To make those improvements, at least start writing things down. And then as I mentioned before, if you can automate anything that’s low risk, things that you think will not cause problems, as I mentioned before, doing a deployment to development environments, definitely automate those things. If you’re deploying to production and you can create some governance, some guardrails around that, you should flag those things as high risk and not just let an LLM make the decision because they will go on side quests, they will hallucinate, they will make mistakes. LLMs are not a deterministic system, so they will execute the same request in different ways every time you ask it, with the same context passed to it. So it’s important to understand the things that LLMs are good at doing and the things that they are not good at doing. Do not confuse a deterministic system like a script or something that executes the same way every time, like the code that we write, it will execute the same way. An LLM will not execute the same way. And you have to really understand the difference between those systems and how you can make them work together safely.

Swapnil Bhartiya: What about those teams that have been on the journey, they adopted AI agents and they also know that they are exposed. What is the fastest way for them to start closing this governance gap?

Jonah Kowall: Well, first you have to admit that you have a gap that you need to fill. And then the first thing is to start writing down a policy. So you know, write down the policy, get agreement that you want to solve that and then figure out how you actually would implement those guardrails. There’s some great open source tools and commercial tools out on the market to help you with that. Defining policy as code is something that we really believe strongly in, where you can define that and manage it just like the rest of your code. And that’s definitely a great starting point. But writing it down is always the first step.

Swapnil Bhartiya: Right now let’s talk about two different teams. Engineering teams that write code and security teams. Security leaders who actually are responsible for the mess that is there. How should these teams split responsibility for governing agentic AI in infrastructure? So at least we know where the bugs stop, somebody is responsible. Because sometimes what happens though, we love to say that it’s everybody’s problem, but when something becomes everyone’s problem, it actually becomes no one’s problem because you assume that hey, that person, that team is doing that. So how do we kind of split responsibility there?

Jonah Kowall: I mean I think it’s always the platform team that sets the governance framework and obviously they work with the security team to validate that, to make sure that there’s not any gaps there. So I think it’s about teamwork. But there’s clearly an owner, the platform team that’s building the automation, that’s creating those guardrails for the rest of the organization and making sure that they’re implemented in the automation and the pipeline is key if that’s what you want to do. But obviously I think any team can take initiative for writing those things down, whether it’s a platform team or a security team, or even an architecture team. We’ve seen all of those patterns. And as long as you really start writing it down, start thinking about how you implement these things and plan to put those guardrails in place, I think you’ll be in much better shape.

Swapnil Bhartiya: So far we have been talking about this survey, what SpaceLift found, but can you talk about what role does SpaceLift play or can play to bridge this gap?

Jonah Kowall: Yeah, so we are an infrastructure as code platform and we handle a lot of different languages, obviously Terraform, but we also handle Pulumi and other open source languages and bring that together. We’re pretty opinionated. We believe GitOps is the right workflow everything should be defined as code should go through a pull request, a review process, and the platform itself is built around that. We also use open Policy Agent OPA, which is an open source policy agent that allows you to define your policy as code. And we built a lot around that. We do self service and a bunch of other things, but the policy and governance and the workflow and automation is the main reason that people come to us is they really want to transform the way that they manage infrastructure. Do it code first, do that review. We also have some amazing AI features and capabilities and we’re going to have a big announcement before Kubecon and RE invent, so stay tuned for that. Some great new products that are coming out.

Swapnil Bhartiya: Jonah, thank you so much for coming back and walking us through what this year’s data is really telling us and also more importantly, sharing some insight how teams can prepare themselves. Because these reports are only as good as people know what they’re doing, but actually do something without action, they don’t matter much. So thank you for all that. And folks of course go and check out SpaceLift and their infrastructure automation report at the same time the work they’re doing in space. Jonah, once again, thank you for talking to me and I look forward to chat with you again soon. Thank you.

Jonah Kowall: Likewise. Thank you.