By AI product teams are routinely scoping infrastructure budgets against compute pricing alone. The costs that breach those budgets, regional data egress and ingress fees, data sovereignty compliance requirements, audit scope obligations, and storage modeling tied to retention rules, are rarely in the initial model and are not visible in a standard cloud bill line item.
In this interview on TFiR, Ari Weil, VP Product Marketing at Akamai, walks through how business and technology leaders should evaluate the full cost of deploying AI applications, starting from user geography and ideal customer profile data and working through architecture, compliance, and scaling decisions.
Guest: Ari Weil, VP Product Marketing at Akamai
Show: TFiR
Here is what every platform engineer, AI product leader, and FinOps practitioner needs to know.
Technical Deep Dive
Q: What should technology and business leaders evaluate beyond raw compute pricing when managing AI costs?
Ari Weil, VP Product Marketing at Akamai, argues that the starting point is not compute at all but rather the core use cases for the application and where users are located. Once those are established, leaders can map compute costs by region, identify applicable data privacy laws, and model egress and ingress fees that accumulate when serving users across borders. Compliance obligations, including audit scope, data sovereignty regimes, and rules governing where decisions can be made and how long data can be retained, all carry direct cost implications for storage and database architecture that must be scoped before deployment.
“What are the costs of compute in those regions that I’m trying to serve? What are the nuances that I need to understand about data privacy laws, data egress and ingress fees that I might be accumulating when I want to do business in that region?” — Ari Weil, VP Product Marketing, Akamai
Q: How should product teams use ideal customer profile data to inform AI infrastructure decisions?
Weil points out that most product teams already work with user stories and ICP definitions that include demographic and firmographic data about target users. That same data, refined through tech previews, alpha releases, or beta programs, directly informs where infrastructure must be provisioned and what regional cost and compliance variables apply. Treating ICP and target account data as an infrastructure input rather than a purely commercial artifact closes the gap between go-to-market planning and architecture decisions.
“Once you know where your users are and those use cases that you’re modeling, it is fairly straightforward for you to understand what are the costs of compute in those regions that I’m trying to serve.” — Ari Weil, VP Product Marketing, Akamai
Q: How do data sovereignty and privacy regimes affect AI application architecture?
Sovereignty and privacy regimes impose hard architectural constraints, not just legal obligations. Weil explains that these regimes specify where data must be persisted, where inference decisions are permitted to occur, and how long data may be retained, all of which directly shape storage architecture, database selection, and model deployment topology. Teams that discover these constraints after initial architecture is set face costly refactoring and potential regulatory exposure.
“Privacy regimes or sovereignty regimes may need to specify how my architecture is set up, where I need to persist data, where I’m allowed to actually make decisions based on the data that I’ve been collecting, how long I can save that data.” — Ari Weil, VP Product Marketing, Akamai
Q: How should teams model AI application growth and capability expansion when planning infrastructure?
Weil identifies two distinct growth vectors teams must plan for separately: scaling compute capacity to handle more requests against existing LLMs or SLMs, and deploying additional or different models as application capabilities expand. These are not the same cost or architectural problem. Whether new capabilities will be consumed by the same user population in the same patterns, or by new segments with different latency and compliance profiles, determines how the microservices layer must scale and how model serving infrastructure should be organized.
“Do I need to continuously deploy more models for these capabilities, and are they going to be consumed by all of the same users the same way?” — Ari Weil, VP Product Marketing, Akamai
Q: How does FinOps apply to AI infrastructure and what is changing in that discipline?
Weil notes that FinOps practices built for cloud infrastructure are evolving to accommodate AI-specific cost variables, including per-query costs, model-specific billing by partner, and the cost structure of AI request-response call patterns. The foundational FinOps discipline of mapping what you have to where users are and what it costs to connect them applies directly to AI workloads, but the inputs and unit economics differ enough that teams need updated frameworks rather than direct carryover of existing cloud cost tooling.
“FinOps teams have been helping cloud practitioners understand what they’re architecting and what sort of costs they’re incurring, giving way to some new versions of FinOps that now are taking into account AI considerations.” — Ari Weil, VP Product Marketing, Akamai
Q: What is the core framework for connecting AI cost decisions across infrastructure, compliance, and users?
Weil distills the full cost modeling exercise into three questions: what do you have, where are your users, and what does it cost to connect the two. This framing treats AI deployment as a microservices architecture scaling problem, which brings established engineering and FinOps tooling to bear rather than requiring entirely new disciplines. The raw materials exist; the discipline is in applying them systematically to AI-specific architecture and compliance variables.
“It becomes really a microservices architecture scaling exercise. The decisions ultimately will be, what do I have? Where are my users? And what does it cost for me to make those two things connect?” — Ari Weil, VP Product Marketing, Akamai
Resources & Documentation
- Akamai, cloud and edge infrastructure platform for AI application delivery and global compute
***
👇 Click to Read Full Raw Transcript
Swapnil Bhartiya: For those businesses and technology leaders who are trying to manage AI costs, what should they be evaluating beyond raw compute pricing?
Ari Weil: Yeah, I think, I mean, the raw compute is certainly important. Understanding if I expect my model to grow and how much data that it might be sending out from various queries is also important. But I would argue you could start simpler than that and just say, what are the core use cases for my app? Where do I anticipate my users to be? My most companies, most product teams are used to looking at user stories that define an ideal customer profile. You’ve got demographic and firmographic information about who those users are that you’re looking to attain. If you’ve done a tech preview, an alpha or a beta release, you’ve probably got an idea of how you might need to fine tune some of those assumptions around ICP and target account lists and things of that nature. But once you know where your users are and those use cases that you’re modeling, I do think that it’s fairly straightforward for you to understand what are the costs of compute in those regions that I’m trying to serve. What are the nuances that I need to understand about data privacy laws, data egress and ingress fees that I might be accumulating when I want to do business in that region? And then you can start to ask yourself, how often do I have to think about things like being audited and what might be in scope for that audit, what sort of privacy regimes or sovereignty regimes I may need to sign up for, which might specify how my architecture is set up, where I need to persist data, where I’m allowed to actually make decisions based on the data that I’ve been collecting, how long I can save that data, because all of that will have an impact for storage modeling and database modeling. And then you need to think about how am I thinking of growing the capabilities of this application over time and what does that need to grow? Do I just need additional hardware so that I can process more off of the existing LLMs or SLMs that I’m managing? Do I need to continuously deploy more models for these capabilities, and are they going to be consumed by all of the same users the same way? And so to me, it becomes really a microservices architecture scaling exercise. And between the finops teams that have been helping cloud practitioners understand what they’re architecting, what sort of costs they’re incurring, giving way to some new versions of FinOps that now are taking into account AI considerations, build by partner, and also how you would architect the AI request response calls as people are getting more examples and more comfortable with AI applications. We’ve got a lot of the raw materials for this, but the decisions ultimately will be, what do I have? Where are my users? And what does it cost for me to make those two things connect?
