Ari Weil, VP Product Marketing at Akamai, explains that centralized GPU clusters were designed for batch or infrequent processing and perform well in that context. The breakdown happens when an application requires an answer before the user, device, or downstream system has moved on, such as a retail personalization engine responding to a click, a fraud detection system keeping pace with financial transaction frequency, or a logistics platform routing first responders in real time. In all of these cases, the physical distance between the centralized compute and the requesting device introduces latency that the use case cannot absorb. The centralized model handles the fan-out and fan-in steps reasonably well but fails hardest on the last mile back to the device.

“The answer can’t be a continent or a country away, it needs to be nearby.” — Ari Weil, VP Product Marketing, Akamai

Weil identifies three categories that illustrate the constraint clearly. First, real-time personalization in retail and e-commerce, where a user expects an application response at the moment of interaction regardless of whether that is a web app, mobile app, or agentic platform. Second, fraud detection, which must operate at the pace and frequency of financial transactions themselves, leaving no tolerance for round-trip latency to a distant data center. Third, voice agents and logistics coordination for first responders, where the system must deliver routing decisions and manage traffic events while the responder is already in motion. In each case, something is actively waiting on the answer and the wait cannot be long.

“If I think about a voice agent or something that is doing real time logistics to send first responders to where they need to go, these are all things where something is waiting on that answer.” — Ari Weil, VP Product Marketing, Akamai

Weil describes the agentic pattern as fanning out to gather tools, data, and context, then fanning back in to synthesize a response. In a centralized architecture this entire loop executes near the data center, but the result still has to travel back across the network to the device that originated the request. That final hop is the last mile, and it is where centralized models accumulate the latency they cannot shed. The fan-out and fan-in steps benefit from proximity to storage and compute resources, but they do not eliminate the distance penalty on the return trip to the user or device.

“The heavy part of that equation is that last mile of reaching the device that made the request in the first place.” — Ari Weil, VP Product Marketing, Akamai

Weil states that the requirement driving the stack selection was the need to run inference across an edge footprint capable of consistently delivering low single to double digit millisecond response times. The NVIDIA hardware and enterprise software stack was chosen because it could support those SLAs when deployed across Akamai’s distributed network. The goal was not simply to place compute at the edge but to do so with a stack that could repeatedly hit the latency targets the real-time use cases demand.

“The reason that we built Akamai Inference Cloud using the Nvidia hardware stack, using the Nvidia enterprise software stack, was that we needed to run inference across an edge footprint that supported low single to double digit millisecond response times.” — Ari Weil, VP Product Marketing, Akamai

Weil explains that Akamai achieves its latency SLAs by distributing inference across its high-performance network and placing points of presence near where requests originate. This means inference does not need to travel to a central location; instead, the computation happens close to the requesting device or system. Security considerations are also applied at these edge points of presence rather than being deferred to a central location, keeping them from adding latency to the critical path of the real-time response.

“The way that Akamai has found that we are able to repeatedly hit those types of SLAs is by federating things out across our high performance network and making sure that there are points of presence to respond to those requests nearer to where the requests are being made.” — Ari Weil, VP Product Marketing, Akamai

Weil draws a clear architectural boundary: persistence, logging, and other workloads that do not block a real-time response belong on centralized compute, where slower and heavier processing is acceptable. Real-time inference for robotics, physical AI, voice agents, fraud systems, and human-facing applications belongs at the edge where latency cannot be tolerated. The design principle is to keep anything that would get in the way of the real-time answer away from the critical response path and let centralized infrastructure handle the non-time-sensitive remainder.

“Leaving that heavier weight, slower centralized compute to be things like persistence and logging and the sorts of things that are not going to get in the way of that real time answer.” — Ari Weil, VP Product Marketing, Akamai