By Infrastructure built around centralized GPU clusters was optimized for one workload: pre-training large generative models. That workload is no longer the dominant driver of AI compute demand. Inference is. And inference at the scale of ubiquitous AI, embedded in every application, every agent, every digital interaction, has fundamentally different latency, distribution, and cost requirements that centralized data centers cannot satisfy.
In this interview on TFiR, Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer at Akamai, breaks down why the centralized model fails as AI shifts from training to inference and from intentional chatbot sessions to always-on AI agents embedded across every digital surface.
Guest: Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer at Akamai
Show: TFiR
Here is what every platform engineer and AI infrastructure architect needs to know.
Technical Deep Dive
Q: Why is the centralized AI data center model the wrong approach for the next phase of AI?
Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer at Akamai, argues that the brute-force approach of concentrating large amounts of compute in centralized locations is both expensive and structurally incapable of achieving the scale that AI will require as it becomes ubiquitous. The core problem is a mismatch between where infrastructure is built and where AI demand is actually going. Blumofe frames this through two specific demand shifts: the move from training to inference, and the move from chatbots to AI applications and agents.
“This sort of brute force approach of building out large amounts of infrastructure in centralized locations ultimately can’t achieve the scale that’s going to be needed as AI moves into its next phase.” — Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer, Akamai
Q: What is the difference between training and inference demand, and why does it matter for infrastructure decisions?
Blumofe distinguishes training, specifically the pre-training of large generative models like LLMs, as the workload for which centralized, dense GPU infrastructure was purpose-built and genuinely well-suited. Training is a mandatory upfront investment. All actual business value, however, is realized through inference. As AI matures, the dominant infrastructure demand is shifting to inference, which has different scale, latency, and distribution requirements that centralized architectures are not designed to handle.
“Training is simply an investment that we have to make to realize the return that you get through inference.” — Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer, Akamai
Q: How does the shift from chatbots to AI applications and agents change infrastructure requirements?
In the chatbot era, AI was a destination: users deliberately navigated to a specific interface to interact with it. In the AI application and agent era, AI becomes a layer embedded in every digital interaction, from booking a healthcare appointment to browsing a car website to composing a message on a desktop. Blumofe describes this as the shift to ubiquitous AI, where the model is no longer intentional and discrete but continuous and ambient. That change in consumption pattern fundamentally alters what infrastructure must deliver.
“Once you move into AI powered applications and AI agents, it becomes ubiquitous. It’s no longer a specific destination, it’s just part of everything that you do.” — Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer, Akamai
Q: What happens to performance and user experience if AI inference remains centralized at ubiquitous scale?
Blumofe draws a direct historical analogy to the early web era, when the internet earned the nickname “the World Wide Wait” due to centralized bottlenecks that could not keep pace with demand. He warns that if AI inference remains centralized as AI becomes embedded in every application and interaction, the industry risks creating an equivalent bottleneck, one he labels “large language molasses.” The implication is that centralized inference at ubiquitous scale produces latency and throughput failures that degrade every downstream application.
“We risk sort of revisiting the old World Wide Wait. It could turn into large language molasses.” — Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer, Akamai
Resources & Documentation
- Akamai, distributed cloud platform for compute, security, and content delivery at the edge
***
👇 Click to Read Full Raw Transcript
Swapnil Bhartiya: We are seeing massive investment in centralized AI data center right now. Why do you believe this centralized everything approach is the wrong model for the future of AI?
Dr. Robert Blumofe: So it’s a great question and ultimately the central thesis is simply that this sort of brute force approach of building out large amounts of infrastructure in centralized locations ultimately, well, it’s expensive, but ultimately even that expense aside, can’t achieve the scale that’s going to be needed as AI sort of moves into its next phase that you might characterize as ubiquitous AI. And I think it’s worth maybe highlighting a couple of ways in which the demand has changed, because ultimately you need to look at the demand and see how the infrastructure aligns to that, that demand. And I would focus maybe on two shifts. One would be the shift from training to inference, and the other one I would characterize as the shift from sort of the early days of a chat bot to an AI application or an AI agent. You know, it wasn’t that long ago focusing on the first of those shifts. It wasn’t that long ago that most of the infrastructure demand really came from the training use case where you were. And by and large I’m talking about the pre training of large generative models like LLMs that was driving a huge amount of the infrastructure demand. And in that use case, absolutely centralized, large scale, dense GPU infrastructure makes a whole lot of sense. But as you move into inference, it changes a lot. And of course, and I think we all know this, that training is really sort of a mandatory cost that is necessary to realize the value through inference. All the value in AI comes from the inference. And training is simply an investment that we have to make to realize the return that you get through through inference. And now as we’re moving into a more mature phase, much more of the demand is coming from inference. And that’s a good thing because again, that’s where we get the value. So inference driving demand is a very, is a very good thing. And I would argue that the nature of inference is changing quite a bit. And again, that’s the shift I’m talking about from the chatbot to the AI application or the AI agent. You know, in the case of the chatbot, I think we really thought of AI as sort of a destination. It was intentional. You went to chatgpt.com to use AI or you fired up your anthropic Claude desktop to use AI. It was intentional. It was a destination. Once you move into AI powered applications and AI agents, it becomes ubiquitous. It’s no longer a specific destination, it’s just part of everything that you do, certainly everything that you do online. You go to a website to look for a car. AI you go to a healthcare provider to make an appointment to see your doctor. AI. Everything that you’re doing is AI powered, probably even everything that you’re doing on your desktop, even irrespective of the web, you know, you want to send something to, to your kids. AI. So AI becomes ubiquitous. That changes the nature of the demand. And in that world where AI is ubiquitous, being used all the time by everyone, a centralized approach just really isn’t, isn’t going to cut it. And we risk sort of revisiting the old. You know, back then we called it the world Wide Weight. It could turn into large language molasses, for lack of a better term.
