Guest: Saiyam Pathak (LinkedIn)
Company: vCluster Labs
Show Name: KubeStruck
Topic: Kubernetes, Cloud Native

Everyone’s talking about AI SRE and AI agents. But according to Saiyam Pathak, Head of Developer Relations at vCluster, the real story is happening one layer below: in the GPU infrastructure and inferencing optimization that makes those AI systems possible in the first place.

The hype around AI site reliability engineering and autonomous agents is undeniable. Projects leveraging AI agents and protocols like Model Context Protocol (MCP) are multiplying across the Cloud Native Computing Foundation (CNCF) ecosystem. But Pathak sees a different trend accelerating even faster: the exponential growth in GPU demand.

“We’re seeing AI farms being built in the US and other countries,” Pathak explains. “Companies beyond NVIDIA are creating their own interesting architectures for better inferencing.” This isn’t just about raw compute power. It’s about solving the efficiency puzzle at the inferencing layer — the moment when trained models actually deliver predictions and responses in production.

Two areas are drawing the most innovation energy right now. First is inferencing optimization, including KV caching and page retention strategies that work natively with Kubernetes. Second is the maturation of Kubernetes itself to handle AI workloads. With version 1.34, Dynamic Resource Allocation (DRA) reached general availability, giving teams a native way to manage GPU and accelerator resources without custom workarounds.

“We need to see how natively LLMs can help build troubleshooting mechanisms inside the Kubernetes cluster,” Pathak notes. The promise of AI SRE — using AI to monitor, diagnose, and fix infrastructure problems — has been discussed at multiple KubeCon events. But it hasn’t quite landed yet. “People are trying, but it’s not there yet,” he admits. Still, early implementations are providing enough value in generating troubleshooting metrics and surfacing insights to keep teams experimenting.

What makes vCluster’s position interesting is that the company isn’t building AI solutions directly. Instead, it operates at the infrastructure layer that powers those solutions. As AI workloads demand more isolation, better multi-tenancy, and faster provisioning, virtual Kubernetes clusters offer a compelling answer without the overhead of managing separate physical clusters.

For platform teams evaluating where to focus their AI infrastructure investments, Pathak’s advice is clear: look beyond the hype of AI agents and focus on the foundation. GPU availability, efficient resource allocation, and inferencing optimization are where outcomes get decided. The AI SRE dream will only materialize once the infrastructure layer is solid enough to support it.