By AI inference is no longer running in a single controlled data center. As workloads distribute across edge nodes, cloud regions, and on-premises infrastructure, compliance teams in regulated industries are confronting a structural problem: existing audit, logging, and recoverability requirements were written for deterministic systems. Non-deterministic AI does not fit that model, and the governance gap is widening faster than regulatory frameworks can close it.
In this interview on TFiR, Ari Weil, VP, Product Marketing at Akamai, walks through how enterprises should approach AI governance across distributed inference environments, which industry verticals are adopting decentralized inference fastest, and why no universal governance framework currently exists.
Guest: Ari Weil, VP, Product Marketing at Akamai
Show: TFiR
Here is what every platform engineer, compliance architect, and enterprise AI team needs to know.
Technical Deep Dive
Q: Why is AI governance for distributed inference harder than traditional IT governance?
Ari Weil, VP, Product Marketing at Akamai, explains that AI governance is converging iteratively, in the same way AI architectures and workloads themselves are evolving. The core difficulty is that traditional governance frameworks were designed for deterministic systems where outputs are predictable, auditable, and reproducible. When enterprises introduce non-deterministic AI into workflows that carry compliance obligations, the scope of what the AI is permitted to affect, and the depth of audit trail required, becomes a direct regulatory liability. Governance cannot be applied uniformly across industries because the regulatory burden, acceptable risk thresholds, and operational maturity differ sharply by sector.
“It’s coming together iteratively, like so much of the architecture, so much of the workloads are as well.” — Ari Weil, VP Product Marketing, Akamai
Q: What compliance challenges do regulated industries face when deploying non-deterministic AI?
Weil identifies industries including education, government, banking, insurance, and trading as operating under a fixed set of compliance rules that govern what data can be logged, how audit trails must be maintained, and what recoverability requirements apply. The problem is that non-deterministic AI, by definition, does not produce the same output for the same input, which makes demonstrating compliance attestation structurally difficult. Enterprises in these sectors must define explicitly how much operational scope they will allow AI to have inside workflows that carry audit obligations, and document that decision as part of their compliance posture.
“When something is non-deterministic, how much scope do I allow it to have inside of that very scoped and audited workflow that I am responsible for attesting compliance to?” — Ari Weil, VP Product Marketing, Akamai
Q: What are the two main categories of AI governance guardrails enterprises need to define?
Weil describes two distinct guardrail categories. The first is compliance guardrails, which are imposed externally by regulators and define what must be logged, audited, and attested. The second is risk management guardrails, which are defined internally and govern how much AI autonomy an organization will accept in its development and production pipelines. These two categories are not interchangeable: compliance guardrails are non-negotiable, while risk management guardrails are a business decision that reflects the organization’s tolerance for shipping velocity versus human oversight.
“One set of guardrails is compliance guardrails. Another one is purely from a risk management basis.” — Ari Weil, VP Product Marketing, Akamai
Q: How are enterprises currently handling human review of AI-generated code in production pipelines?
Weil describes a wide spectrum of approaches currently in use. At one end, some organizations are publicly reducing human review of AI-generated code and allowing AI systems to build their own test harnesses to evaluate the code they have written. At the other end, many enterprises consider this approach unacceptable and require human review before any AI-generated code reaches production. The position an organization takes on this spectrum reflects both its risk tolerance and its competitive pressure to ship frequently. Neither position is industry-standard, and no consensus framework currently dictates where the line should be drawn.
“Some companies are famously going on podcasts talking about how everything is AI-coded, and they are allowing AI to build its own test harnesses to evaluate the code that it has written. There are many businesses that think about that and find it a horrifying scenario.” — Ari Weil, VP Product Marketing, Akamai
Q: Which governance frameworks are currently being used to map AI risk in enterprise environments?
Weil references NIST and MITRE ATT&CK as frameworks that enterprises are applying to AI risk mapping. NIST provides a structured approach to quantifying and measuring risk, while MITRE ATT&CK helps organizations identify where specific checks and balances are needed across their systems. Akamai’s global services team works directly with customers on AI architecture reviews, testing scenarios, scaling strategy, and penetration testing as organizations move AI workloads into production. Weil is explicit that these are starting points, not comprehensive solutions, because well-established governance frameworks specific to autonomous AI do not yet exist.
“There have been frameworks like using NIST, like mapping things to MITRE ATT&CK to really understand where you feel like you need a given set of checks and balances.” — Ari Weil, VP Product Marketing, Akamai
Q: What is the current state of regulatory readiness for autonomous AI governance?
Weil states that regulators are only beginning to form their positions on autonomous AI, and are currently leaving most governance definition to individual companies. The consensus view across the industry, in his assessment, is that enterprises are not yet ready for fully autonomous, self-contained AI systems that move from concept to production without human interaction. The amount of human interaction required, what humans are actually reviewing, and what audit and reporting obligations apply are all being defined company by company rather than through any centralized regulatory mandate.
“Most people say we are not ready for a fully autonomous and self-contained AI concept-to-production deployment right now without human interaction.” — Ari Weil, VP Product Marketing, Akamai
Q: Which industries are the earliest adopters of decentralized AI inference?
Weil identifies media and technology companies as the fastest-moving early adopters, driven by a long history of adapting to digital disruption. The media sector’s specific requirements around high-fidelity video capture at 4K and 8K resolutions, rapid transcoding, anomaly detection, social and streaming clip generation, and multilingual speech-to-text and text-to-speech processing at global scale are creating strong demand for distributed compute and inference capabilities. Gaming is a second vertical moving quickly, and retail and commerce are accelerating as well, with agentic web experiences driving new applications of edge computing and serverless functions toward inference workloads.
“Media has famously, as they have kept pace with all sorts of digital disruptions, had to continuously reinvent themselves, and we are seeing this now from the way that video needs to be captured in very high fidelity, processed very quickly, and streamed back out at global scale.” — Ari Weil, VP Product Marketing, Akamai
Q: How is physical AI and robotics development connected to gaming and media infrastructure?
Weil describes a direct lineage from neural network applications in gaming and immersive media to physical AI and robotics use cases in manufacturing, healthcare, and defense. Organizations that built early versions of metaverse environments and immersive gaming experiences developed neural network and physical AI expertise that is now being reapplied to optimize assembly lines and support healthcare applications. AR and VR technologies originally developed for gaming are also being brought into real-world applications in defense and other sectors, applying the same underlying infrastructure to entirely different purposes.
“People that have cut their teeth on creating early versions of the metaverse and immersive gaming are now starting to think about how they can optimize assembly lines and healthcare.” — Ari Weil, VP Product Marketing, Akamai
Q: How is the retail and commerce sector applying edge inference to agentic web experiences?
Weil explains that the web is being redefined in retail and commerce to be more agentic, moving beyond static recommendations engines toward AI-driven personalization that mimics human-like interaction. The technology foundation being applied to this shift is the same edge computing and serverless function infrastructure that has been in use for years, now redirected toward inference workloads. Retailers are seeing these inference applications reach production quickly and scale rapidly, driven by consumer expectations for personalized experiences and by the competitive opportunity that early movers are identifying.
“The end consumer has always looked for human-like personalization and recommendation, and the early versions of it are really starting to show people that there is a big competitive opportunity.” — Ari Weil, VP Product Marketing, Akamai
Resources & Documentation
- Akamai, edge computing, distributed inference, and global services for AI workload architecture and security
- NIST AI Risk Management Framework, structured framework for quantifying and managing AI risk in enterprise environments
- MITRE ATT&CK, adversarial tactics and techniques framework used to map AI system risk and identify required checks and balances
***
👇 Click to Read Full Raw Transcript
Swapnil Bhartiya: Let’s now also talk about governance. How should enterprises think about governance when inference is running across more distributed environments?
Ari Weil: So the interesting thing about governance is that it’s coming together iteratively, like so much of the architecture, so much of the workloads are as well. You know, it’s easy to say that you should have a human in the loop. It’s easy to say that you should buy things like API gateways and AI gateways and start thinking about know testing, harness evolutions and how you should be evaluating Claude code and having people review that before it goes into production environments. But I think the reality is, is that across the industry, the governance question is really going to be very situational and specific to the business, to their maturity and to the environment that they exist in or are selling into. So for example, if I’m part of a traditionally heavily regulated industry, like the education industry, government financial services like banking, insurance or trading, there are a given set of rules that I have to abide by. There’s a certain burden that I have of what I can log and how I provide audit trails and recoverability and things of that nature to all of my data. And the challenge that I’m facing right now in the AI era is when something is non deterministic, how much scope do I allow it to have inside of that very scoped and audited workflow that I am responsible for a testing compliance to? And I think that’s one set of guardrails, compliance guardrails. Another one is just purely from a risk management basis. How much am I going to allow people to write code and run code from conception into production with whatever level of human guardrails I have or without? I mean, some companies are famously going on podcasts right now talking about how everything is clogged, coded, and they’re less and less reviewing it with human reviews. And they’re allowing AI to build its own test harnesses to evaluate the code that it’s written. There’s many businesses that think about that and it’s a horrifying scenario. And other businesses that look at that and say that’s just the cost of doing business today because of the need to ship and ship so frequently. I think when it comes to governance, the same sort of rules apply to how I think about quantifying risk for my business, how I measure that risk for my business, and how I start to think about introduction of new technologies, especially when they’re autonomous. I don’t believe that we have well established frameworks for this, but you know, there have been frameworks like using nist, like mapping things to mitre, ATT and CK to really understand where you feel like you need a given set of checks and balances. Akamai works with a lot of our customers, for example, through our global services team on this architecture, on testing scenarios, on how they can think about scaling and penetration testing and other things like that that they have to evaluate as they move to production and evolve their applications. But when it comes to governance, I don’t think that there’s a broad brush approach to what we need to do other than most people saying we are not ready for a fully autonomous and self contained AI, you know, sort of concept to production deployment right now without human interaction. The amount of human interaction, what the humans are actually doing, what you have people auditing, you know, reviewing and reporting on, I think is very specific to the business. And regulators, as we’ve seen, are just now starting to kind of get their ideas together and they’re leaving a lot of this to the individual companies to define.
Swapnil Bhartiya: Are there any specific industries where you see this shift towards decentralized inference is getting moment or makes more sense or they are the early adopters of this idea.
Ari Weil: We definitely see some of our typical early adopters from a media and technology perspective adopting things quicker. And I think that that’s part of the overall industry and their approach to technology. Throughout the years, media has famously, as they’ve kept pace with all sorts of digital disruptions, had to continuously reinvent themselves. And we’re seeing this now from the way that video needs to be captured in very high fidelity, you know, 4K, even 8K streams processed very quickly, turned into a derivative work and then streamed back out to people at a global scale without incurring too much latency. Being a huge driver of how you can use different types of compute infrastructure to actually do the transcoding, to do the anomaly detection, to, you know, put together the social clips, the streaming clips, the downloads that people all expect to be able to take speech to text and text to speech and really have that working across a number of different languages and mediums very, very quickly, driving a fair amount of innovation. We’re seeing the same sort of thing around gaming and we’re even seeing a quick evolution of things that were originally conceived of through neural networks. And using physical AI and robotics to support gaming use cases now evolve into other sorts of robotics and physical AI use cases. So people that have cut their teeth on creating early versions of the metaverse and immersive gaming are now starting to think about how they can optimize assembly lines and healthcare and thinking about even modern warfare and the way that AR&VR are currently being brought into the real world as a different technology or a different purpose than what they were originally conceived of. And then we’re seeing from just a rote sort of recommendations engine and being able to mine data and come out with tailored recommendations, new sorts of form factors of the work or the work product being produced. A lot of effort is being put in from the retail and commerce segment where the web is being redefined to be more agentic, to now really needing to take that technology that we’ve done through edge computing and serverless functions and start to apply that more to inference use cases. And we’re seeing those meet, you know, production very, very quickly and then scale up as people are realizing that the end consumer really does. You know, they’ve always sort of looked for this human like, you know, personalization recommendation and the early versions of it are really starting to show people that there’s a big there, there and a big competitive opportunity to be search.
