By AI inference is not a scaled-up version of AI training. When AI moves from periodic training jobs to always-on agents and applications, the demand profile changes entirely: high bandwidth, low latency, continuous, and geographically distributed. Centralized infrastructure was not designed for this workload, and brute-force scaling into hyperscaler regions will not close the gap.
In this interview on TFiR, Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer at Akamai, breaks down why the architectural lessons from the early web scalability crisis and the cybersecurity transformation of the past decade apply directly to what AI infrastructure teams are building today.
Guest: Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer at Akamai
Show: TFiR
Here is what every platform engineer and AI infrastructure architect needs to know.
Technical Deep Dive
Q: How does AI inference infrastructure compare to the early web scalability crisis?
Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer at Akamai, draws a direct parallel between the scalability problems of the early web and the infrastructure demands now emerging from AI inference. In the early web era, applications were centralized in a small number of hosting locations, primarily concentrated in places like Ashburn and San Jose. As demand scaled from static pages to video and dynamic interaction, traversing large distances to reach those centralized locations produced bandwidth and latency failures that led observers to predict the web would collapse entirely.
“History does repeat itself, and it’s kind of repeating itself for the umpteenth time here.” — Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer, Akamai
Q: What made centralized web infrastructure fail as demand scaled, and what replaced it?
Blumofe explains that early web hosting was even more centralized than today’s hyperscalers, and the failure mode was straightforward: bandwidth and latency requirements could not be met when every user request had to traverse large distances to reach a handful of data centers. The solution was not more centralized capacity but a fundamentally different approach rooted in mathematics, distributed systems, and algorithms. Delivering content from the edge of the Internet dramatically increased available bandwidth and reduced latency, which is what made the web viable at scale and what enabled the transition from static sites to dynamic, always-on web applications.
“Tom and Danny came forward with a better approach. Math, algorithms, distributed systems rather than brute force. And they showed that you can actually deliver websites and web applications from the edge of the Internet, dramatically increasing the available bandwidth, dramatically lowering the latency.” — Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer, Akamai
Q: How did cybersecurity infrastructure face a similar centralization failure, and how was it solved?
Blumofe identifies a second historical parallel roughly a decade ago, when cybersecurity threats shifted from relatively unsophisticated actors like Anonymous to sophisticated ransomware operators and powerful DDoS extortion campaigns. Centralized security architectures could not absorb or respond to that threat model. The solution again came from the same distributed systems playbook: moving defense capabilities to the edge of the network using math and algorithms rather than consolidating protection in centralized chokepoints.
“With ransomware and powerful DDoS extortion attacks, the centralized approach just wouldn’t work. And again, you have to borrow from this playbook of math, distributed systems, algorithms.” — Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer, Akamai
Q: How does the demand profile change when moving from AI training to AI inference?
Blumofe draws a clear distinction between the infrastructure requirements of training and those of inference. Training workloads are relatively low-bandwidth and can tolerate high latency. Early inference use cases, such as text-based chatbots operating in a request-and-response pattern, also fit within centralized infrastructure. The demand profile changes fundamentally when AI powers applications and agents, where interaction becomes ubiquitous, constant, high-bandwidth, and latency-sensitive. That shift renders centralized infrastructure architecturally insufficient regardless of how much capacity is added.
“As you move from training to inference, as you move from fairly low bandwidth and high latency types of interactions like the chatbot, you move into an AI powered application or an AI agent. The nature of the demand just changes dramatically.” — Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer, Akamai
Q: Why can’t AI inference be solved by scaling centralized cloud infrastructure?
Blumofe is explicit that brute-force scaling of centralized infrastructure will not meet the demands of AI inference at the application and agent layer. The same structural problem that caused the early web to be called the “worldwide wait” applies here: when demand is globally distributed, constant, and latency-sensitive, routing it through a small number of centralized locations creates irreducible physics-level constraints. The demand has already moved to the edge, and the infrastructure must follow it there.
“The demand has moved to the edge, so the infrastructure and capabilities of AI have to also move to the edge.” — Dr. Robert Blumofe, Executive Vice President and Chief Technology Officer, Akamai
Resources and Documentation
- Akamai, edge cloud platform for AI inference, cybersecurity, and content delivery at global scale
***
👇 Click to Read Full Raw Transcript
Swapnil Bhartiya: If AI training is more or less like writing software, inference is like deploying it globally so people can use requires totally different infrastructure. You have handled high latency sensitive use cases like live sports and global cybersecurity. How do the lessons from those challenges apply to AI infrastructure today?
Dr. Robert Blumofe: Yeah, that’s a great point. And I do see a lot of parallels to the early days of the web.
Swapnil Bhartiya: Also.
Dr. Robert Blumofe: I some of the changes that happened maybe a decade or so ago in cyber security. So in many ways, you know, people say this all the time. History does repeat itself, and it’s kind of repeating itself for the umpteenth time here. So while there’s obvious differences, there’s a lot about what’s happening now that I think does parallel what we saw in the early days of the web. You know, as the web was getting popular, really transforming the Internet, there were a lot of concerns that the web simply wouldn’t scale to meet the demand. And in many ways, those concerns probably were well founded because you did have a situation where web applications were centralized. Now, we were pre cloud, but we did have hosting providers. And arguably the hosting providers back then were even more centralized than today’s hyperscalers. By and large. Most of the infrastructure was, you know, in the U.S. for example, was heavily located in places like Ashburn and San Jose. So every time you used a web application, you had to traverse large distances into a handful of centralized locations. And while that might have been okay in the very early days, where a website was a pretty static thing, just some text, maybe a few images as you move into video, for example, and large demand for that video, you simply cannot meet the bandwidth requirements and the latency requirements through that centralized model. And that’s really, I think, what led people to, you know, jokingly say that the World Wide Web should be, you know, called the worldwide, worldwide wait. And people speculated that the web would simply collapse. And really that concern was the beginning of Akamai, where, you know, Tom and Danny, the two founders, came forward with a better approach. Math algorithms, distributed systems rather than brute force. And they showed that you can actually deliver websites and web applications from the edge of the Internet, dramatically increasing the available bandwidth, dramatically lowering the latency. And really that’s what made the web work. And that was a critical ingredient also, as the web transitioned from these static sites to dynamic, where your communication is happening all the time, it’s not just click on a link and wait for a response. You’re constantly interacting with these web applications. CDNs made all of that work. And a similar approach, really the same approach is in many ways what enabled powerful cybersecurity defenses. Because cybersecurity also went through a pretty strong transformation about 10 years ago, maybe a bit less. Where you moved from our biggest concern being things like Anonymous to sophisticated ransomware and the world of sophisticated attackers. With, with ransomware and powerful DDoS extortion attacks, the centralized approach just wouldn’t work. And again, you have to borrow from this playbook of math, distributed systems, algorithms, and that worked. AI today, I think, is in a very similar regime where as you move from training to inference, as you move from fairly low bandwidth and high latency types of interactions like the chatbot, where you’re simply typing some text, waiting for a response type, typing some text, waiting for a response, you move from that into an AI powered application or an AI agent, The nature of the demand just changes dramatically. It becomes ubiquitous, it’s constant, it’s high bandwidth, it requires low latency. And again, the brute force approach just isn’t going to work. You can’t do this with purely centralized infrastructure. The demand has moved to the edge, so the infrastructure and capabilities of AI have to also move to the edge.
