By Teams moving AI workloads into production routinely underestimate total cost because they model compute and storage but not data movement. Egress fees on most hyperscaler platforms run 9 to 12 cents per gigabyte and are structured to make data portability economically painful. When inference is centralized and results must be returned to users and downstream services across geographic boundaries, those fees compound at scale in ways most cost models never capture.
In this interview on TFiR, Ari Weil, VP Product Marketing at Akamai, breaks down how to model total AI infrastructure cost, why hyperscaler egress pricing functions as a lock-in mechanism, what a neutral and converged architecture offers as an alternative, and how to decide where inference, training, and application activity should be distributed versus centralized.
Guest: Ari Weil, VP Product Marketing at Akamai
Show: TFiR
Here is what every platform engineer and FinOps practitioner needs to know.
Technical Deep Dive
Q: Why do egress costs catch AI teams off guard when they move into production?
Ari Weil, VP Product Marketing at Akamai, explains that most teams focus on the cost line items that hyperscalers and AI platforms actively promote, such as compute and model hosting, while ignoring the full set of costs that emerge at scale. Egress fees, concurrency costs, multi-use-case overhead, and data movement across geographic boundaries are not surfaced prominently but accumulate significantly as user growth and application complexity increase. Weil argues that companies growing successfully over the next 12 to 36 months will be those that modeled the entire bill from the start.
“The companies that are going to grow the fastest and the most successfully will have modeled the whole bill and not just the line item that the hyperscalers are touting.” — Ari Weil, VP Product Marketing, Akamai
Q: What cost factors should teams include when modeling total AI infrastructure spend?
Weil identifies several cost dimensions teams need to account for beyond raw compute: user acquisition cost, concurrency cost from simultaneous users, the number of use cases enabled on the platform, where proprietary intellectual property and systems of record live, and where the application is actively learning from external data sources including customers and industry. Each of these dimensions behaves differently at scale and must be modeled explicitly before committing to an architecture. Ignoring any one of them produces a cost model that breaks under real production load.
“You need to really understand all of the use cases that will incur cost for you as you scale.” — Ari Weil, VP Product Marketing, Akamai
Q: What is the first question teams should ask before choosing a cloud or AI infrastructure partner?
Weil recommends starting with a self-hosted baseline: what would it cost to own all of your data, application code, and infrastructure entirely, whether in your own data center, leased colocation space, or another self-managed environment. Running that calculation first forces clarity on availability zones, data boundaries, and where geographic distribution creates cost exposure. It also gives teams a reference point against which to evaluate what partnering with a hyperscaler actually costs in comparison.
“The first question is: what if you just owned all of your data in your application code? That will have you start to understand availability zones and data boundaries and where distribution can become more expensive.” — Ari Weil, VP Product Marketing, Akamai
Q: How do hyperscaler egress fees function as a data lock-in mechanism?
Weil states directly that 9 to 12 cents per gigabyte in egress fees on most hyperscaler platforms is priced to make moving data out economically prohibitive. News coverage over the past 12 to 18 months has documented how expensive it is for companies to export or download their own data when switching providers or repatriating workloads. Some vendors offer a one-time egress exemption when a customer is leaving their cloud entirely, but Weil cautions that teams should verify they genuinely intend to exit before using that option, as it is not a general-purpose cost reduction mechanism.
“9 to 12 cents a gigabyte is something that is designed to keep your data where it is on most hyperscaler platforms because the cost of moving it is prohibitive.” — Ari Weil, VP Product Marketing, Akamai
Q: What is a neutral architecture and how does it reduce egress exposure?
Weil describes a neutral architecture as one where data is stored in a single canonical location and then federated or synchronized across multiple regions and environments, with retrieval and serving happening wherever it makes the most sense for the application. The goal is to avoid paying for data movement between microservices, across geographic boundaries, and between storage and serving layers by designing the data flow so that movement is minimized or eliminated. This approach reduces vendor dependency and gives teams more control over where cost is incurred.
“A neutral architecture is one that will allow you to store data in one place that might then federate or synchronize it in a bunch of different areas, allow you to retrieve it somewhere else, and then serve it where it makes the most sense.” — Ari Weil, VP Product Marketing, Akamai
Q: How does a converged core-to-edge architecture like Akamai’s change the cost calculus for AI applications?
Weil acknowledges that a converged architecture spanning core to edge can appear expensive at first glance when compared to a centralized deployment. However, when teams account for all the costs they avoid, including data movement between microservices, cross-region transfer fees, and the operational overhead of managing data and application code separately from serving infrastructure, the math can shift into a net cost benefit. The key condition is that teams must have modeled all use cases across the platform rather than evaluating infrastructure cost in isolation.
“When you think about all of the costs that you don’t have to pay from moving data between microservices or across geographic boundaries, it could actually turn into a cost benefit for you if you’ve modeled out all of your use cases.” — Ari Weil, VP Product Marketing, Akamai
Q: How should teams decide where to run inference versus where to run training and fine-tuning?
Weil frames this as the core architectural question: teams need to know where inference runs for their specific application, how often and where pre-training, post-training, and fine-tuning occur, and where the rest of application activity is happening. Once those three variables are mapped, teams can combine them with their user acquisition model, user growth projections, and lifetime value assumptions to determine which parts of the application benefit from centralization and which benefit from distribution. Without that complete picture, architecture decisions default to convenience rather than cost optimization.
“If you know where inference runs, how often you need to do pre and post training and fine tuning, and where the rest of your application activity is occurring, then you can put all of those factors together and think about what sort of architecture you are really focused on.” — Ari Weil, VP Product Marketing, Akamai
Resources & Documentation
- Akamai Cloud Computing, converged core-to-edge infrastructure for AI and application workloads
***
👇 Click to Read Full Raw Transcript
Swapnil Bhartiya: You touched upon egress cost earlier. When teams move AI into production, egress cost often come as a surprise to them. How much of that hidden cost of centralized inference is actually just data movement getting results back to users and applications? How big of a factor is that and how can teams plan for that as well?
Ari Weil: I think moving into the next 12 to 36 months, the companies that are going to grow the fastest and the most successfully will have modeled the whole bill and not just the line item that, you know, the hyperscalers are touting or that all of the different AI platforms are touting. They need to really understand all of the use cases that will incur cost for them as they scale. And that can be anything from what is my user acquisition cost, what is my concurrency cost with multiple users? How many use cases have I enabled on this platform? Where is some of that data that I own completely? My intellectual property, my systems of record? And where am I learning actively and developing from the world or the industry at large and my customer base? Because the first thing that I can do is I can understand what would that cost be if I said I don’t want to use multiple cloud providers or even my principal cloud provider and I want to manage the data in my own environment? My own environment could be my own data center, it could be leased space, it could mean a lot of different things. But that’s the first question of what if you just owned all of your data in your application code that will have you start to understand availability, zones and data boundaries and where distribution can become more expensive if you use your own versus using a hyperscaler’s infrastructure. The second thing is, once you’ve decided to partner with somebody, what would that cost to leave? Because now, if you look at all of the last 12 to 18 months of news around the cloud space, there have been all sorts of news stories about how much does it cost to leave a cloud provider? What does it cost for companies to take over their own data to export or download it themselves? And so the reality is, 9 to 12 cents a gigabyte is something that is designed to keep your data where it is on most hyperscaler platforms because the cost of moving it is prohibitive. Some vendors have said if you’re leaving their cloud altogether, then they will give you a one time exemption where you can download your data. But you should be making sure that you indeed intend to exit that cloud environment if you do so. It’s also given rise to something that people are considering to be more of a neutral architecture. So a neutral architecture is one that will allow you to store data in one place that might then federate or synchronize it in a bunch of different areas, allow you to retrieve it somewhere else, and then serve it where it makes the most sense. So a converged architecture like Akamai, where we have core to edge, can in some cases look like you might be thinking about an expensive distributed architecture on the face of it. But when you think about all of the costs that you don’t have to pay from moving data between microservices or other services, moving it across geographic boundaries, and what it would take to manage all of your data and application code and then serve it as needed on demand with that low latency could actually turn into a cost benefit for you if you’ve modeled out all of your use cases across the platform. And I think that’s the ultimate question here, is where does inference run for your application? How often and where do you need to do things like pre and post training and fine tuning of your model? And then where is the rest of the activity in your application actually occurring? Because if you know those things, and you know how you’re modeling user acquisition and user growth and longtime value, then you can put all of those factors together and think about what sort of an architecture am I really focused on, and which parts of my application can be centralized versus where I’ll benefit from distribution.
