By AI inference cost projections built at the prototype stage routinely collapse in production. The gap is not token pricing. It is the compounding cost of data movement across availability zones, cloud regions, and service boundaries that most teams default to pricing at zero during design. By the time workloads scale, cross-zone transfer fees and internet egress charges have fundamentally changed the economics of the application.
In this interview on TFiR, Ari Weil, VP Product Marketing at Akamai, breaks down how physical inference location, architectural boundary decisions, and egress pricing interact to make AI workloads significantly more expensive than teams anticipate, and why cost modeling must move earlier in the development cycle.
Guest: Ari Weil, VP Product Marketing at Akamai
Show: TFiR
Here is what every platform engineer and AI infrastructure architect needs to know.
Technical Deep Dive
Q: Does the physical location of AI inference affect performance and cost?
Ari Weil, VP Product Marketing at Akamai, confirms that distance in AI inference carries a direct and measurable cost. Every time a request crosses a zone, region, or cloud boundary, teams pay twice: once in latency, measured as time to first token and total token consumption time, and once in data transfer fees that sit entirely outside token pricing. Most teams default to pricing that data movement cost at zero during application modeling because the primary focus is on workflow design and prompt output, not infrastructure economics.
“Every time a request crosses a zone or a region or a cloud boundary, you’re going to pay twice.” — Ari Weil, VP Product Marketing, Akamai
Q: What are the two cost components every AI inference request incurs at a zone or cloud boundary?
Weil identifies two distinct cost hits. The first is latency: time to first token and the total duration to consume all tokens from the request and response. The second is transfer fees, which are separate from token pricing and apply any time data moves between microservices, broader services, or across cloud boundaries. These two costs stack independently and both compound as workloads scale.
“Transfer fees are not part of your token price. It’s what you’re charging anytime that you need to move data either between microservices, overall services or other cloud boundaries.” — Ari Weil, VP Product Marketing, Akamai
Q: What does cross-availability-zone traffic actually cost per gigabyte in an AI workload?
Cross-availability-zone traffic runs approximately one cent per gigabyte in each direction. In an AI workload where a model sends data to GPUs deployed in a different part of the architecture, and that workload also requires retrieving embeddings or calling other services along the way, that one-cent-per-gigabyte charge applies each time data crosses the boundary. The directionality matters: sending and receiving are both billed.
“Cross availability zone traffic can run a penny per gigabyte in each direction.” — Ari Weil, VP Product Marketing, Akamai
Q: What do internet egress fees cost when AI data crosses cloud boundaries to the public internet?
When data moves across cloud boundaries to the internet, egress fees rise to 9 to 12 cents per gigabyte. That is a 9x to 12x increase over cross-availability-zone rates. At application scale, this delta makes AI inference economics materially different from what prototype-stage modeling projects, and is a core reason why AI inference can become more expensive than the human labor it is intended to replace.
“Egress fees if I send them across the Internet are 9 to 12 cents each.” — Ari Weil, VP Product Marketing, Akamai
Q: Is AI inference latency a physics problem or an architectural choice?
Weil draws a clear distinction between latency that is physically unavoidable (the speed of light over distance) and latency that is generated by architectural decisions such as which zones to span, which cloud boundaries to cross, and how services are arranged relative to one another. The latter is billable and controllable. Teams that treat architectural latency as though it were a physical constraint are absorbing avoidable costs and building them permanently into their unit economics.
“Latency and egress, that’s not physics, that’s not the speed of light, that is an architectural choice that you’re billed for as though it were physics.” — Ari Weil, VP Product Marketing, Akamai
Q: What framework should AI teams use to model infrastructure costs before building?
Weil recommends that teams apply 10 to 15 years of distributed cloud application cost thinking directly to AI infrastructure design. The core question is: what does data movement cost me? That discipline, already established for traditional cloud workloads, translates directly to AI. Teams should understand how many zones their application must traverse, where it needs to be geographically placed, and what egress costs look like at production scale before they launch, not after.
“Where people are building AI applications and afraid that they don’t have a frame of reference for what they’re building, they can look at the last 10 to 15 years of building distributed cloud apps and understand what does data movement cost me.” — Ari Weil, VP Product Marketing, Akamai
Q: When should cost modeling enter the AI application development cycle?
Weil argues that cost is consistently treated as a late-stage consideration in AI development, arriving only after architectural and workflow decisions have already been locked in. His position is that cost must be integrated into user testing and scalability testing before the application launches. Waiting until the production launch phase to model egress and zone transfer costs means those costs are baked into the architecture with no practical path to optimization without rebuilding.
“Cost is one of the levers that we treat late in that cycle. I would argue that you have to really bring that into a lot of your user and scalability testings before you launch your app.” — Ari Weil, VP Product Marketing, Akamai
Resources & Documentation
- Akamai, cloud and edge infrastructure platform referenced throughout this discussion on AI inference placement and egress cost optimization
***
👇 Click to Read Full Raw Transcript
Swapnil Bhartiya: Can you talk about what role does physical location of AI inference matter or play a role when it comes to performance latency and most importantly, cost. Because cost is going to become a very big factor because as there are reports coming out, it’s actually cheaper to hire an employee than to use AI.
Ari Weil: That is true. I mean, the first thing that I would start with, and you’re absolutely right about the way that some people are modeling applications, distance does come at a cost, and a lot of times you’re going to price that cost at zero. Because when you’re modeling your application, you’re thinking about things like what is the workflow that I’m enabling? Or what is the output of a given function or a given prompt supposed to be? And really focusing on that. But every time a request crosses a zone or a region or a cloud boundary, you’re going to pay twice. Effectively, the first time you pay, you’re going to pay for latency. How much is my time to first token? What is the overall time that I’m going to expend consuming all of the tokens from my request and response? The other one is the transfer fees that basically are not part of your token price. It’s what you’re charging anytime that you need to move data either between microservices, overall services or other cloud boundaries. And if you think about that latter piece, cross availability zone traffic can run a penny per gigabyte in each direction. So if I’m going to be sending data from a model to a set of GPUs that are deployed in a different part of my architecture, and then as part of that workload, I’m going to be send getting either, you know, some of my embeddings back to store, or if I have to go and call other services along the way, I am going to pay again, penny per gigabyte each time that I send that data across. Egress fees then if I send them across the Internet are 9 to 12 cents each. So there’s a sort of tension between how do I build my application to take full advantage of the cloud capabilities that I know how to build on, or the different providers are making available, does it make sense for me to keep my workload on a given cloud or where I have to cross cloud boundaries, am I ready to incur that 9 to 12 cents per gigabyte, understanding how my app needs to scale so things like latency and egress, that’s not physics, that’s not the speed of light, that is an architectural choice that you’re billed for as though it were physics. The physics is what you have to think about. And again, where people are building AI applications and afraid that they don’t have a frame of reference for what they’re building, they can look at the last 10 to 15 years of building distributed cloud apps and understand what does data movement cost me, and then really start to think, to your point, when I move from a prototype to production, how many zones do I have to move from and to where does my application need to be? And one of the levers that we treat late in that cycle, late in that consideration phase, is cost. And I would argue that you have to really bring that into a lot of your user and scalability testings before you launch your app.
