By AI inference workloads are not uniform, but most infrastructure strategies treat them as if they are. Deploying every inference job to a centralized GPU cluster introduces latency, raises cost, and ignores the geographic reality of where demand actually originates. The gap between compute location and request origin is a structural problem that worsens as AI adoption scales globally.
In this interview on TFiR, Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer at Akamai, covers how Akamai AI Grid addresses the placement and composition challenge for distributed AI inference across 4,000 edge locations worldwide.
Guest: Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer at Akamai
Show: TFiR
Here is what every platform engineer and AI infrastructure architect needs to know.
Technical Deep Dive
Q: What is Akamai AI Grid and what does it actually do for AI inference workloads?
Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer at Akamai, describes AI Grid as intelligent orchestration built around one core principle: delivering the right infrastructure in the right place at the right time. The system does not route every workload to the same compute type or the same location. Instead, it evaluates each inference request and matches it to the appropriate combination of GPU, CPU, storage, and connectivity across Akamai’s 4,000 edge locations.
“It’s the right infrastructure in the right place at the right time. There is no one size fits all.” — Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer, Akamai
Q: Why is a single GPU cluster not sufficient for all AI inference use cases?
Blumofe is direct on this point: massive GPU clusters serve some workloads well, but they are not appropriate for all of them. CPU-only deployments cover other scenarios but also fall short in different cases. The actual requirement is a workload-specific mix of GPU, CPU, storage, and connectivity. Forcing every inference job through a single infrastructure type introduces inefficiency, latency, and cost that cannot be solved by scaling that one type further.
“It’s not a one size fits all. It’s not massive GPU cluster for everything. That’s good for some use cases.” — Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer, Akamai
Q: How does geographic placement of compute affect AI inference performance?
Blumofe frames location as a first-class architectural variable, not a secondary concern. If demand originates in Dallas, Texas, routing inference to a data center in Ashburn, Virginia adds unnecessary latency and distance. AI Grid resolves this by placing infrastructure at the location of demand and also accounting for proximity to any tools or data sources the workload needs to access during inference.
“You can’t do everything in Ashburn, Virginia. It’s deploying the right infrastructure in the right place for that particular use case.” — Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer, Akamai
Q: How does connectivity factor into edge placement decisions for AI workloads?
Connectivity is treated as a required component of the infrastructure decision, not an assumption. When an inference workload depends on tools or resources distributed across multiple locations, the placement decision must account for network proximity to all of those dependencies, not only proximity to the end user. Blumofe names connectivity explicitly alongside GPU, CPU, and storage as a variable in the orchestration logic.
“You need connectivity to all of those things. So it’s the right infrastructure in the right place at the right time.” — Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer, Akamai
Q: What can AI infrastructure architects learn from how cloud providers solved the heterogeneous compute problem?
Blumofe credits cloud providers for establishing the foundational idea that infrastructure does not have to be uniform. The ability to select a specific mix of CPU, GPU, and storage per workload is something cloud excelled at and something AI infrastructure must carry forward. His critique is not of the heterogeneous model itself but of the centralization constraint: cloud compute remains geographically concentrated in ways that distributed inference at the edge is designed to correct.
“I think the cloud has done such a great job at this hybrid notion of infrastructure. They’re more centralized than we’d like them to be, but in terms of delivering the right type of infrastructure, that’s one of the things that cloud has really excelled at.” — Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer, Akamai
Resources & Documentation
- Akamai, platform provider for AI Grid and distributed edge inference orchestration across 4,000 global locations
***
👇 Click to Read Full Raw Transcript
Swapnil Bhartiya: Akamai recently launched AI grid intelligent orchestration for distributed inference across your 4000 edge locations. What exactly is it and what does it mean in practice for developers?
Dr. Robert Blumofe: Yeah, thanks for the question. It’s a great question. I would raise it to simply the basic notion of providing the right infrastructure in the right place at the right time. So starting with right infrastructure, it’s not a one size fits all. It’s not, as we said, it’s not massive GPU cluster for everything. That’s good for some use cases and it’s not just cpu. CPU again alone is good for some cases, but again, not for everything. And it’s not just storage. It’s the right combination of gpu, CPU and storage and connectivity for the use case. So that’s the first question, is you have to deliver the right infrastructure for the use case as it’s presented to you at that time. Then there’s the where. Again, it’s not a one size fits all. You can’t do everything in Ashburn, Virginia. Um, it’s, it’s, it’s deploying the right infrastructure in the right place for that particular use case. If the demand is coming from, from, from Dallas, Texas, infrastructure in Dallas, Texas, if it’s using tools that are distributed in other locations, you want to have proximity to those tools. And then there’s the connectivity. You need connectivity to all of those things. So it’s the right infrastructure in the right place at the right time. There is no one size fits all for, for this stuff. And that’s a challenge, by the way, because, you know, it’d be nice if we could simply invest in a particular kind of infrastructure in a particular location. Problem solved. And it’s just not going to work that way. It hasn’t worked that way for the web. And that’s certainly, by the way, I think the cloud has done such a great job at this hybrid notion of infrastructure. I think that’s one of the really great things about cloud is that it’s not a one size fits all. Now. They’re more centralized than we’d like them to be. But I think in terms of delivering the right type of infrastructure, I think that’s one of the things that the cloud has really excelled at. You can choose what you’re getting, the mix of CPU to GPU to storage, so that you don’t have to be stuck in that one size fits all. And I think that’s the key, probably the key challenge. But the key recipe for success is recognizing that it’s the right infrastructure, the right place at the right time. It’s not a one size fits all. Not easy, but that’s what needs to be delivered.
