Guest: Saiyam Pathak (LinkedIn)
Company: vCluster Labs
Show Name: KubeStruck
Topic: Kubernetes, Cloud Native

GPU infrastructure is expensive. Organizations are investing millions in NVIDIA hardware but struggling with utilization and security. vCluster just announced a partnership with NVIDIA that changes the equation entirely. The solution covers the complete multi-tenancy spectrum and enables organizations to maximize GPU utilization while maintaining the isolation and security AI workloads demand.

vCluster announced a significant partnership with NVIDIA at KubeCon CloudNativeCon North America. The partnership brings vCluster’s virtual cluster technology to NVIDIA DGX infrastructure with reference architectures that combine private nodes, auto nodes, and vNodes for complete isolation.

Saiyam Pathak, Head of Developer Relations at vCluster, explains the core problem this partnership addresses. Organizations want their AI infrastructure built securely. They want data to stay local. They want to inference their own documents and code within their own infrastructure while using LLM capabilities. NVIDIA provides the GPU hardware. Kubernetes provides the orchestration layer. vCluster makes it work with proper multi-tenancy.

The Multi-Tenancy Spectrum

vCluster spent 2024 building features to cover the entire multi-tenancy spectrum. Previously, vCluster only supported shared nodes. You could create a virtual cluster with a hosted control plane, but workloads still used the host infrastructure. The new private node concept changes this completely.

With private nodes, you maintain a hosted control plane pod on the host cluster but join physical nodes outside the host cluster to virtual clusters. This brings complete isolation for teams that need maximum security. Then came auto nodes with Karpenter integration baked in. vCluster is the only solution providing Karpenter for bare metal or any Kubernetes cluster.

Pathak emphasizes this point. Traditional bare metal setups have limited capacity. You have CPU nodes and GPU nodes. Different teams want their own Kubernetes clusters, but each cluster needs three to four nodes minimum. You cannot create that many clusters with limited bare metal capacity.

The vCluster approach is different. Combine everything into a single Kubernetes cluster from your bare metal hardware with CPUs and GPUs. Create virtual clusters and distribute them to teams. With private nodes and auto nodes enabled, you get maximum security plus efficient resource utilization. The Karpenter integration for bare metal uses free nodes for specific virtual clusters and releases them when not needed.

For organizations requiring even more security without privileged pod access, vNodes provide the answer. The complete architecture looks like one cluster, virtual nodes, virtual clusters inside vNodes, with auto nodes enabled.

Beyond Just Another Kubernetes Cluster

The challenge with AI infrastructure goes beyond simply bringing Kubernetes to GPUs. GPU architectures you get from cloud providers differ from what you purchase directly from NVIDIA. The networking layers work differently. Setting up NVIDIA drivers and making GPUs available for pod consumption is challenging.

The standard Kubernetes scheduler is not efficient for batch workloads or gang scheduling. You need solutions like Kueue scheduler or RunAI, which NVIDIA recently acquired. These components integrate natively with vCluster. The partnership ensures all building blocks work together properly. It is not just a Kubernetes cluster but a cluster that can actually run and scale AI workloads.

AI SRE and Infrastructure Innovation

Pathak sees two key areas for innovation. First is the inferencing space. Making sure KV caching, page retention, and related mechanisms work properly with Kubernetes. Kubernetes 1.34 brought Dynamic Resource Allocation to GA, but the ecosystem needs to see how LLMs can help natively and build troubleshooting mechanisms inside clusters.

AI SRE has been discussed since previous KubeCons, but nobody has it completely right yet. The space provides enough value for troubleshooting metrics and results, but it needs more work. This makes AI infrastructure particularly interesting. vCluster is not building AI solutions directly but sitting on the infrastructure layer that powers those solutions.

Platform Evolution

vCluster has evolved significantly in 2024. The company moved from polishing existing features to launching entirely new capabilities. The tenancy spectrum model itself represents a new way of thinking about tenancy. The latest release includes Istio integration and standalone vCluster, where vCluster can become the host cluster with virtual clusters created on top.

This entire stack can power DGX machines and become private AI infrastructure. The reference architectures with NVIDIA provide organizations with proven patterns for deployment. GPU utilization becomes maximized. Security and isolation requirements are met. Teams get the Kubernetes experience they need without the overhead of managing separate clusters.

The partnership positions vCluster as infrastructure for the AI infrastructure wave. Organizations building internal AI capabilities need hardware, orchestration, and proper multi-tenancy. NVIDIA provides the first. Kubernetes provides the second. vCluster completes the picture with the third.