By Multi-agent AI systems do not behave like single-shot request-response applications. A single agentic task can require 100 to 200 round trips across models, APIs, and external tools, and each hop adds latency that compounds across the entire workflow. When users, data, and third-party systems are geographically distributed, routing all of that traffic through a centralized cloud region does not just add delay. It structurally undermines what the agent is trying to do.
In this interview on TFiR, Jon Alexander, SVP of Product, Cloud Technology Group at Akamai, breaks down why centralized cloud architecture is a poor fit for real-world agentic workloads and what the distribution problem actually looks like at the infrastructure level.
Guest: Jon Alexander, SVP of Product, Cloud Technology Group at Akamai
Show: TFiR
Here is what every platform engineer and AI infrastructure architect needs to know.
Technical Deep Dive
Q: What actually breaks when multi-agent AI workflows run on centralized cloud infrastructure?
Jon Alexander, SVP of Product, Cloud Technology Group at Akamai, draws directly from his Internet architecture background to explain the failure mode. Unlike a single-shot query, an agentic workflow loops through a multi-step process, potentially 100 to 200 iterations, calling different models, pulling data from APIs, and executing generated code at each step. When all of that traffic is routed through a centralized cloud region while users, data, and third-party tools are distributed globally, latency compounds at every hop and the system stops functioning effectively for real-world applications.
“A centralized system is not going to be the optimal location to run all of those components of the solution.”
Jon Alexander, SVP of Product, Cloud Technology Group, Akamai
Q: How is a multi-agent AI system architecturally similar to a microservices deployment?
Alexander frames agentic AI the same way he frames microservices: a decomposed system where multiple specialized components must coordinate to achieve a specific business outcome. In a microservices architecture, discrete services each have a separate purpose and must be orchestrated together. In an agentic workflow, different tools, different models, and different interaction steps play the same role. No single component delivers the outcome alone, and the coordination overhead across all of them determines whether the system performs or fails.
“In an agent you have kind of a similar concept where you have multiple components of the system, different tools, different models that you’re talking to, and different interactions at different steps of the process which are all required to achieve the outcome.”
Jon Alexander, SVP of Product, Cloud Technology Group, Akamai
Q: Why does distributed data and distributed users make centralized AI cloud a worse fit for agentic workloads?
Centralized cloud performs well when data, users, and models are co-located near the same region. In practice, that is rarely the case for enterprise agentic deployments. Users are geographically distributed, the data those agents access is distributed, and the third-party APIs and tools the agent calls are themselves distributed across different systems. Routing all of those interactions through a single centralized cloud region adds latency at every call, and because agentic workflows make many calls per task, the compounding effect is severe enough to make the architecture impractical for production use.
“Users are distributed, data is distributed, and often the systems that you’re talking to, the APIs that you’re calling, the tools that you’re calling, these third party systems, they’re distributed as well.”
Jon Alexander, SVP of Product, Cloud Technology Group, Akamai
Resources & Documentation
- Akamai Cloud Technology Group, distributed cloud and edge infrastructure for enterprise AI and application workloads
***
👇 Click to Read Full Raw Transcript
Swapnil Bhartiya: When multi agent workflows are forced through traditional centralized cloud architecture, what actually starts to break?
Jon Alexander: As we move from relatively simple question answer type interactions with AI and we’re moving into agentic workflows, you’re moving into a multi step looping process where the agent is making multiple requests to the model and that is a multi step process. I personally think my background is in Internet architecture, so I think about this as a lot like a microservices type deployment where an application is built with decomposed into all these different components and all of these pieces have a separate purpose that needs to be coordinated to achieve a specific business outcome. In an agent you have kind of a similar concept where you have multiple components of the system, different tools, different models that you’re talking to, and different interactions at different steps of the process which are all required to achieve the outcome. So it’s not a single shot where I just fire off one response, get one answer and deliver that back to the user. I go around maybe 100 times, maybe 200 times, I call these different systems, I pull in data from APIs, I jet generate code that I might execute. All of these are multiple steps and depending upon where these are running, it can compound the latency. Now, a centralized cloud can work well if all of the data, if all of the users, if the models, if everything is located close. Often what we’re finding is that isn’t the case. Users are distributed, data is distributed, and often the systems that you’re talking to, the APIs that you’re calling, the tools that you’re calling, these third party systems, they’re distributed as well. And this all pulls us back to a centralized system is not going to be the optimal location to run all of those components of the solution. And so in these multi agent workflows, we’re often finding that they don’t work well for real world applications where distribution of data, of users and the tools exist.
