In an era where AI adoption is accelerating across enterprises, the need for robust infrastructure that can both scale efficiently and implement responsible guardrails has become paramount. At KubeCon + CloudNativeCon in London, Sudeep Goswami, CEO of Traefik Labs, shared insights on how the company’s partnership with Akamai is addressing these critical needs through specialized AI Gateway technology.

Traefik Labs’ Role in the AI Revolution

When asked about Traefik Labs’ core business, Goswami explained that the company provides “essential building blocks to enterprises who are deploying AI and API infrastructure,” including “an application proxy, an API gateway, an AI gateway, and the full API management life cycle.”

This focus positions Traefik Labs at the intersection of two converging worlds. “The world of AI and the world of APIs are blending together, because AI models are being exposed and consumed as APIs,” Goswami noted. “The more AI models, the more APIs. The more APIs, the greater the need for API management.”

Defining Scalable and Responsible AI

Goswami broke down the concept of scalable AI in practical terms. As enterprises bring AI models in-house—whether on-premises or in cloud environments like Akamai’s—scaling efficiently becomes a challenge.

“The scalable part comes into play when you ask, how do you do this at scale across the Akamai Cloud,” he explained. Traefik Labs has introduced semantic caching functionality that enables more efficient use of computational resources. “For certain AI models and certain questions, if there’s already a response available and someone asks a similar question again, you don’t need to rerun the costly CPU and GPU cycles.”

This approach allows intelligence to be pushed “all the way to the edge” even in places where GPUs aren’t available, optimizing both performance and cost.

Regarding responsible AI, Goswami emphasized it’s about “controlling the inputs and outputs of data going into and out of the models.” Traefik Labs has launched a technology called Content Guard that allows administrators to establish policies for handling sensitive information.

“When you are feeding data into an AI model, not all data is the same,” he explained. “If you have sensitive data, you don’t want to bring that into the model.” Content Guard can redact specific types of data or block entire requests if they contain too much sensitive information.

Real-World Use Cases for Responsible AI

While many organizations talk about responsible AI in theoretical terms, Goswami outlined concrete implementation approaches. The first step many enterprises take is bringing AI models in-house to run them in a “controlled, kind of like a walled garden,” keeping data confined within their environment.

However, as Goswami pointed out, “maybe that’s not enough, because you don’t know what the LLM — what the model — actually holds.” He illustrated this with a practical example: “Let’s say a law firm is turning all of its documents and knowledge into a database, and they’re using AI to train it… But still, you don’t want the AI model to accidentally give out some sensitive information about another case.”

This is where Traefik Labs’ approach of “trust, but verify—trust, but have controls in place” becomes crucial, providing “an additional layer of protection” even within controlled environments.

Traefik Labs’ Position in the AI Infrastructure Stack

When asked about where Traefik Labs fits into the AI implementation layers, Goswami was clear: “We are providing the AI gateway component of it, so we become the guardrail for anything going in and out of these AI models. Therefore we can add that layer of control, governance, and security.”

This positioning leverages Traefik Labs’ existing expertise in API management. As Goswami explained, the company’s experience allowed them to “move fast and introduce an AI gateway, which is really a specialized case of our API gateway that we’ve already had.”

The Akamai Partnership: Expanding Scale and Reach

The partnership with Akamai brings complementary strengths to both companies. “What we bring is the tech, the knowledge, and the community of people who have embraced open source, API gateways and API management,” Goswami said. “What Akamai brings is their extensive network, their architecture—a highly distributed core architecture—and the thousands of PoPs that they have.”

This collaboration creates value as “Akamai brings more customers to run their AI models on the Akamai cloud,” while Traefik Labs “helps secure those AI models, exposes them as APIs and gives them that control in the governance layer.”

Structure of the Partnership and Customer Benefits

The partnership is designed to benefit customers of both companies. According to Goswami, “It’s going to be both ways, because what we have built is very agnostic.”

This agnostic approach is critical in today’s complex IT landscape. “It’s not a one-cloud world. It’s a hybrid cloud — also on-prem… and there’s the edge,” Goswami emphasized. Traefik Labs provides “infrastructure that is agnostic to those environments, so you can run it in multiple places depending on the use cases and needs.”

This flexibility is particularly valuable in today’s geopolitical environment, where data sovereignty and compliance issues vary across regions. Goswami explained that their solution “gives customers an on-ramp and an off-ramp… you’re not tightly coupled to the environment. You are decoupled from it.”

This means that organizations can quickly adapt to changing requirements: “At any given point, if you want to off-ramp from it and go to another environment, you can do so — again, being cloud-native, Kube-native as the foundation.”

Conclusion

As enterprises accelerate their AI adoption, the partnership between Traefik Labs and Akamai represents a significant development in the infrastructure landscape. By combining Traefik Labs’ API governance capabilities with Akamai’s global network reach, the companies are addressing critical needs for scalability, security, and responsible implementation of AI technologies.

Guest: Sudeep Goswami (LinkedIn)
Company: Traefik Labs
Show: KubeStruck