By Running AI on Kubernetes exposes a hard gap: the scheduler was built for stateless web workloads, not long-running GPU jobs that need specialized hardware, survive control plane restarts, and scale to zero between inference bursts. Teams hitting that gap are patching around it with custom operators and manual workarounds instead of relying on platform primitives that simply work.
In this interview on TFiR, Dipesh Rawat, Kubernetes v1.37 Release Lead at CNCF, walks through how watch cache resiliency, HPA scale to zero, checkpoint/restore, and Dynamic Resource Allocation signal a deliberate shift toward a workload-aware Kubernetes designed to meet AI infrastructure demands.
Guest: Dipesh Rawat, Kubernetes v1.37 Release Lead at CNCF
Show: TFiR
Here is what every platform engineer running AI workloads on Kubernetes needs to know.
Technical Deep Dive
Q: Is Kubernetes actually ready for AI workloads in production?
Dipesh Rawat, Kubernetes v1.37 Release Lead at CNCF, says the readiness question has shifted. Features like watch cache resiliency, HPA scale to zero, and checkpoint/restore are not coincidental additions. They reflect direct input from the AI community and a project-wide intent to make Kubernetes workload-aware rather than a generic declarative scheduler. The graduation pipeline from alpha and beta to stable is the mechanism that makes these features safe for wide production adoption.
“All those features are giving us a hint as to where Kubernetes is going in the future releases.”
Dipesh Rawat, Kubernetes v1.37 Release Lead, CNCF
Q: What does “workload-aware” mean in Kubernetes and why does it matter for AI?
Rawat draws a direct contrast with today’s model: Kubernetes takes a pod with declarative resource requests and schedules it. Workload-aware means the platform asks what type of workload it is, whether it needs specialized hardware, and what to do if that hardware is unavailable. For AI teams, this closes the gap between generic infrastructure and the specific demands of GPU jobs, long-running training runs, and inference workloads that scale to zero between requests.
“Instead of you giving us a pod and we just taking that resource and just scheduling it, Kubernetes on the whole is trying to be workload aware.”
Dipesh Rawat, Kubernetes v1.37 Release Lead, CNCF
Q: How does Dynamic Resource Allocation (DRA) help teams manage specialized AI hardware in Kubernetes?
Rawat cites DRA as the key mechanism for handling specialized hardware in the cluster. It gives teams a structured way to request hardware beyond standard CPU and memory, such as GPUs and accelerators, and to define how those resources are managed and allocated. This removes the need for custom workarounds and makes hardware lifecycle part of the Kubernetes control loop.
“We are taking inputs from the community to bring those features such as DRA, to have specialized hardware in the cluster. How do you request those and how do you manage those?”
Dipesh Rawat, Kubernetes v1.37 Release Lead, CNCF
Q: How does Kubernetes control plane resilience protect long-running AI jobs?
Long-running training jobs expose a specific failure mode: if the control plane goes down under load, the job is lost. Rawat explains that control plane resilience improvements in v1.37 are explicitly designed to prevent this. Watch cache resiliency is part of that work, ensuring the cluster remains stable even when large, sustained workloads are running against it.
“We are also making the control plane more resilient and more secure so that if long running and big jobs are running, the cluster does not go down because of that.”
Dipesh Rawat, Kubernetes v1.37 Release Lead, CNCF
Q: Which Kubernetes v1.37 features are moving to stable and what should production teams prioritize?
Rawat says multiple features currently in alpha and beta are being evaluated against graduation criteria for promotion to stable. Stable status is the signal for production teams: it means the API is locked, the feature is safe to depend on, and it will be supported going forward. He does not name specific graduation timelines but confirms the pipeline is active and AI-relevant features are included.
“The features that are already in alpha and beta, based on their graduation criteria, we are looking to graduate them to stable so that they are more production ready and can be used widely by the teams.”
Dipesh Rawat, Kubernetes v1.37 Release Lead, CNCF
Resources & Documentation
- CNCF, Cloud Native Computing Foundation, home of the Kubernetes project and release process
- Dynamic Resource Allocation (DRA), Kubernetes official documentation for requesting and managing specialized hardware
***
👇 Click to Read Full Raw Transcript
Swapnil Bhartiya: Now if we put all of these three together, watch cache resiliency, HPA scale to zero and of course checkpoint restore, what does that tell us about where Kubernetes is heading, where the focus is? It’s quite obvious AI is the biggest workload these days. Though initially people used to say that AI is just another workload on Kubernetes or Linux. But no AI is changing. Everybody’s, you know. So talk a bit about what does it tell about Kubernetes and its focus.
Dipesh Rawat: Just like you said, that all those features are giving us a hint as to where the Kubernetes is going in the future releases. So what I would like to say it’s kind of a coincidence that the features that we are building is also helping the AI community and also we are taking inputs from the community to bring those features such as dra, to have specialized hardware in the cluster. How do you request those and how do you can manage those? So I would say that more such features are in the pipeline and the features that are already in alpha and beta, based on their graduation criteria, we are looking to graduate them to stable so that they are more production ready and can be used widely by the teams. So I would say that all in all, Kubernetes as a whole, we are trying to make it workload aware. So instead of you giving us a pod and we just taking that resource, declarative resource and just scheduling it, Kubernetes on the whole is trying to be workload aware. What type of workload is it? Does it require some type of new hardware? If that hardware is not present, what do we do? How do we manage that hardware? So all in all I would say yes, it is going to help AI workloads and at the same time we are also making the control plane more resilient and more secure so that if long running and big jobs are running so the cluster does not go down because of that.
