Cloud Native

How to Do Kubernetes Fleet Management at 1,000 Clusters | Hong Wang, Akuity | TFiR

0

GitOps made syncing a change fast, but that change still has to reach hundreds of Kubernetes clusters, and teams often do not know which cluster goes first, what to check between batches, or when to stop. The Argo CD User Survey found about two thirds of teams still promote with scripts or in-house tools. Reviewing every pull request stops working the moment one release targets 1,000 clusters.

In this interview on TFiR, Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO at Akuity, walks through how fleet management handles batches, failure thresholds, verification and AI agents across hundreds of Kubernetes clusters.

Guest: Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO at Akuity
Show: The Agentic Enterprise
LinkedIn: Hong Wang
Watch the discussion on YouTube

Here is what every platform engineer needs to know.

Technical Deep Dive

Q: How do you manage and promote changes across hundreds or 1,000 Kubernetes clusters?

Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO at Akuity, says AI demand is pushing teams from 5 or 10 clusters to 20 or 25, and some customers run 1,000. A conference keynote described Meta managing about 100 large GPU clusters for AI training and serving with Argo, Kargo and Terraform. The tools in this space were designed for a handful of clusters, which makes deploying and managing at large scale a new challenge.

“So originally you have maybe 5, 10 clusters. Right now you’re about 20, 25. Even for the big guy, Meta, 100.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: Why do pull request reviews and traditional tooling break down at fleet scale?

With 3 to 5 clusters, a human can review every change in a pull request before it reaches production. Nobody can review 1,000 PRs a day, so teams end up merging blindly and the review step loses its purpose. At that scale Wang says teams need a tool that gives visibility, governance and control, and he names two gaps with no out-of-the-box answer: progressive delivery and failure tolerance.

“Can you review 1,000 PR every day? … you don’t review it, you just blindly merge it.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: Why is failure normal in large fleets, and when should a rollout stop automatically?

In a fleet with many GPUs, failures come from power, networking and other causes, so some failure is expected. Fleet management lets a team set a failure tolerance, for example 3%. Under it, the rollout finishes and the team goes back to look at the failed clusters. Over it, the rollout rolls back or stops before the failure spreads, without waiting for a human to step in.

“…the failure is normal in the fleet scale. It’s not normal when you only have 5 cluster.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: How do you roll out to 1,000 clusters in batches instead of all at once?

Pushing to 1,000 targets at once means an incident or bug leaves all 1,000 unhealthy at the same time. The alternative is batches, such as 50 at a time, with extra verification and validation before the next batch starts. Akuity ships this as core fleet management functionality.

“You may want to do 50 at a batch, but you also wanna do the additional verification, validation before you go to the next batch.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: What is a target in Kargo Enterprise and how does it organize a fleet?

Argo CD is the foundation that deploys to a single environment, while Kargo handles higher-level orchestration and was designed for promotion from day one. Akuity adds an abstraction called a target, a logical grouping that makes each cluster a first-class citizen. The view shows which of 1,000 targets are healthy, behind the release or in an error state.

“…in one environment at dev, you can have 5 targets. In production, you can have 400 targets.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: How is updating across environments different from updating within a fleet?

An environment is a type, such as dev, staging or production. Dev may have 5 environments and staging 6 to 10, but enterprise customers can run 500 to 1,000 in production. That is the point where management has to work at the level of individual targets.

“…when we look at our enterprise customers, their production environment is 500 to 1,000. It’s not uncommon.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: How does Kargo verify each stage before a promotion moves on?

Kargo has a verification concept built into promotion as a gate. After a new binary runs in dev, verification can trigger functional tests or query an observability solution for golden signals, and a pass certifies the artifact for the next stage. Staging can run more demanding checks such as stress tests, and the same gates work for 5 or 500 targets.

“…you deploy some functionality or new binary into a dev environment and it’s running, but it’s running is not good enough for you to consider to put it into the staging or production.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: How do you keep new clusters from adding manual orchestration?

Adding a sixth cluster to five is easy, but adding another 10 or 20 is not. Akuity uses a fleet concept and a target set concept: label the new cluster correctly and it joins the fleet and its fleet-level template. What needs to be deployed is provisioned in no time, and the fleet can grow and shrink as clusters come and go.

“You get a new cluster, you label the cluster in the right way, it will join the fleet.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: What real-world fleet use cases exist, from AI training to retail stores?

Wang says Meta used Terraform for everything, but a single-line configuration change took 4 hours to roll out. Terraform stays for core infrastructure, while Argo and Kargo handle system and business applications, cutting deployment to several minutes. Retail customers run Kubernetes in 1,000 to 3,000 franchise stores, and vendors facing customer demand for security boundaries may run 100 dedicated clusters for 100 customers.

“Especially in AI, time is money. And the cluster is very expensive with all the GPU running there.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: How do neoclouds and AI infrastructure providers manage fleets that grow this fast?

CoreWeave is publicly an Akuity customer, and Akuity has powered and managed its back-office fleet for 3 to 4 years. Wang says GPU resources are still scarce and scattered, so teams building AI infrastructure source clusters from AWS, Google, Azure and neoclouds. That creates multicloud, multi-region fleets, and Akuity’s distributed agent architecture manages clusters wherever they run, even on a laptop.

“…the GPU resources actually still scarce. And it’s actually also very scattered.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: Why does fleet automation matter when GPU time is so expensive?

Wang points to per-GPU-hour pricing on Amazon, which could be $20 or even more. A strong automation tool that manages infrastructure effectively and delivers software fast and reliably saves money. That is Akuity’s positioning.

“Just check what’s the per GPU per hour price from Amazon as an example. It could be at $20 or even more.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: How does AI fit into environment promotion, and what governance does it need?

Wang says Akuity works at a mission-critical lower layer, so automation has to be right before AI can help much. Akuity made the Kargo portion open source so it can become an open standard whose schema and workflow any AI can understand. The Agentic Control Plane, announced in September, exposes context through the MCP protocol and adds a governance layer for what AI can and cannot do.

“…the key thing is we need to get the automation right. Then AI can be more helpful.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: Where do AI agents help with promotion, and where should humans stay in the loop?

AI agents will be the next stage of infrastructure management, but adoption is slow for good reason: bad behavior in production means instant revenue loss. With sandboxes, controls, rich context and stronger models, teams are getting comfortable with read-only access to production data. They are then starting to let agents modify production systems, which is why AI SRE solutions are showing up.

“With the sandbox, with the control, with the rich context, with the stronger model, I think that’s definitely starting changing.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: Are AI agents already working with production infrastructure safely?

Akuity’s Agentic Control Plane, launched in September, is already used by many customers. Akuity acts as the context provider for whoever runs an AI agent, and as the governance layer for whoever wants an agent to act on infrastructure. That layer includes unified authentication, governance control policy and auditability for day-to-day operations.

“Who is the one who want to taking action against that infrastructure? We providing the governance layer for people to taking action there.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: What should an AI agent be allowed to do when it promotes a change?

Wang says AI should have read-only access to all production systems at the bare minimum, and agents should be able to promote changes in the future. The gap is deletion and more sophisticated operations. He would let AI start on tasks such as auto-memory and performance tuning, while the scarier operations need more muscle first, and he expects the answer may change within 6 months.

“I do think the AI should have the read-only access at the bare minimum to all the production system, no doubt.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: How do you deliver and manage the lifecycle of AI agents themselves?

Akuity wrote a blog on AI agent delivery with OpenShift and Kargo as an experiment and demo, which drew about 50,000 LinkedIn impressions and 500 reactions. Wang sees the AI agent as a new kind of workload after bare metal, VMs, containers, microservices and serverless. Teams will run 100, 1,000 or 10,000 agents, so lifecycle management becomes a fleet problem.

“You are not going to running one agent. It’s more likely about 100, 1,000, or 10,000 agents.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: What should platform teams watch next as AI agents become a new workload?

Wang says KubeCon gives early insight into what workloads will run on infrastructure next, and he believes the AI agent is the next big thing. New work such as OpenShift and agent substrate is emerging, and Akuity plans to spend more time understanding it and building a customized solution. Adoption that once looked like 3 years away now happens in 6 months.

“Things are moving so fast now, originally thinking about, oh, 3 years someone will be adopted. Nowadays it’s 6 months.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Q: What is next on the roadmap for fleet management and agent lifecycle?

Wang says Akuity will double down on fleet management because infrastructure keeps growing aggressively. On the agent side, he sees a lot of innovation at the runtime level that Akuity wants to incorporate. How to deploy agents and manage their lifecycle will be part of its story next year.

“…we want to double down, triple down on the fleet management again.”Hong Wang, Co-Creator of Argo CD and Kargo, Co-Founder and CEO, Akuity

Resources & Documentation

  • Fleet Management for Kargo Enterprise, Akuity blog post announcing fleet management for Kargo Enterprise and releasing to many targets
  • Kargo Custom Steps, Akuity blog post on custom steps for GitOps promotion in Kargo
  • Argo CD, the GitOps foundation for deploying to a single environment
  • Kargo, the orchestration layer for promotion across stages and targets

***

👇 Click to Read Full Raw Transcript

Swapnil Bhartiya: Platform teams, as we all know, aren’t running just a few Kubernetes clusters anymore. They’re running fleets across regions, clouds, and customers. GitOps made syncing a change fast and faster, but then that change has to reach hundreds of clusters, and teams don’t know which goes first, what to check between batches, or when to stop. So they fall back on scripts and runbooks. The Argo CD User Survey found about 2/3 of teams still promote with scripts or in-house tools. Environment promotion, as we all know, has become the hard problem, and teams are looking for answers. And we have answers. Akuity has been working on this problem, and today we have with us once again Hong Wang, co-creator of Argo CD and Kargo, and co-founder and CEO of Akuity. It’s good to have you back on the show.

Hong Wang: Yeah, thank you for having me again.

Swapnil Bhartiya: It’s my pleasure. As I was kind of mentioning, a lot of teams still promote changes with scripts. And in-house tools, we have talked about, of course, Argo CD at scale and promotion with Kargo before last discussion. So from where you sit, what has changed for platform teams since then and what hasn’t changed?

Hong Wang: Yeah, I think it’s a great question. I want to recall one thing is yesterday in the keynote, close to the end, the Meta team, the Facebook team was on the stage talking about how they managing about 100 clusters, big GPU clusters doing the training and the serving of the AI. And their solution was Argo, Kargo, and Terraform. That was in the keynote. Okay. I think that’s kind of like a great example, very live example to mentioning that due to the AI demand, Actually, people has ever-increasing infrastructure to manage. So originally you have maybe 5, 10 clusters. Right now you’re about 20, 25. Even for the big guy, Meta, 100. I think that’s just like a small portion of their infrastructure. But we have even some customers, they have like 1,000 clusters. So that actually putting a very unique challenge is how do you deploy and manage those clusters in a large scale? Even for the big guy, Meta, 100, I think that’s just like a small portion of their infrastructure. But we have even some customers, they have like 1,000 clusters. So that actually putting a very unique challenge is how do you deploy and manage those clusters in a large scale? I think the traditional tools, we are more designed especially in the open source side, is we are thinking about more like a handful

Swapnil Bhartiya: of clusters versus when you look at somebody working at Meta scale, what actually changed and Today’s tools, where they fall short, and then we talk about how Akuity is solving those problems.

Hong Wang: Exactly. So that’s a great question. So basically when you have like 3 or 5 clusters, you can like treat them very carefully. Means every change you can doing the human review, you want to do the pull request before it goes to the production or multiple production, but it’s kind of very handful. Means the human can keep up. But when you look at, oh, you want to deploy the same software to 100 or 1,000 location, Can you review 1,000 PR every day? So in the end, it’s like you don’t review it, you just like blindly merge it. But in the end, it’s okay. When you’re not blindly merging it, what’s the point of creating those PR or that human process? In the end, you actually need a more sophisticated tool to managing the fleet in a large scale. So in the end, it’s like you don’t review it, you just like blindly merge it. But in the end, it’s okay. When you’re not blindly merging it, what’s the point of creating those PR or that human process? In the end, you actually need a more sophisticated tool to managing the fleet in a large scale, give you the visibility, give you the governance, give you the control. I think there are 2 small but it’s like key problem we wanna address in the fleet management. One is actually more like a progressive delivery. The reason is when you have 1,000 target or 1,000 cluster, you don’t want to deliver software to 1,000 location at one time because when you have some incident or bug, that means your all 1,000 target is unhealthy at the same time. So you wanna actually do in batches. The reason is when you have 1,000 target or 1,000 cluster, you don’t wanna deliver software to 1,000 location at one time because when you have some incident or bug, that means your all 1,000 target is unhealthy at the same time. So you wanna actually do in batches. You may want to do like 50 at a batch, but you also wanna do the additional verification, validation before you go to the next batch. That’s what we are providing as a fleet management solution, as a core functionality. Another thing is actually when you have a big infrastructure, like even in the AI side, actually you have so many GPU. So when you have a certain number, the failure is inevitable due to whatever reason, due to the power, due to networking, due to all the reason. So means the failure is normal in the fleet scale. It’s not normal when you only have 5 cluster. That means in the Fleet Management, you have a certain concept called like failure tolerance is when you can set in the 3% as an example. If it’s under 3%, you want to finish the rest. You want to come back to look at why that less than 3% failed and babysitting and charging them. But when it’s over 3%, you actually want to automatically roll back or maybe stop there before it’s propagated to more cluster. Before the human involved. I think that’s kind of the 2 things are very key on the fleet scale, that there’s no out-of-the-box solution solving that, and which we are providing now.

Swapnil Bhartiya: And so how is, you know, how is of course Akuity solving that problem for

Hong Wang: them? Yeah, I mean, if you go back to about what we’re doing here at Akuity, so we’re providing the software delivery platform, enterprise version of it, and the Argo CD is the foundation, which we are providing now. And so how is, you know, of course, Akuity solving that problem for them? Yeah, I mean, if we go back to about what we’re doing here at Akuity, so we’re providing the software delivery platform, enterprise version of it, and the Argo CD is the foundation. We are talking about how you deploy the single environment a binary or artifacts to that namespace, but Kargo is focused on a higher-level orchestration. So means our Kargo was designed for the promotion always from day 1. So initially we’re talking about maybe 3 or 4 different stages. That means 3 or 4 different environment. It’s also mapping to 3 or 4 different clusters. But now we are going to do another abstraction, introduce a new thing called the target. Okay. Means in one environment at dev, you can have 5 targets. In production, you can have 400 targets. So means that we kind of like adding the additional logical grouping, actually making the fleet, making the target actually to be the first-class citizen in the field. Means we can give you a really nice view because in anynines, when you’re managing like 1,000 targets, you will be curious about, oh, out of the 1,000, who are great in a good state? Who are behind the release? Who are in the error state?

Swapnil Bhartiya: Let’s say a cert manager, right? And of course, as you gave example, Meta, thousands of clusters, Updating Git is easy, but what of those decisions stay out of tooling versus some of those decisions stay in the tooling? That’s number one part. And second part is that, are you leveraging AI? Because AI makes things work better at scale at this point or not?

Hong Wang: I think since we are more focused on like a lower layer, like a mission-critical layer, I think the AI is there. But the key thing is we need to get the automation right. Then AI can be more helpful. I think what we are trying to do is we are more likely to— because why we open source the Kargo portion is because we want the Kargo to be the open standard. That means any AI can understand our schema, understand our working flow, then they can interact with us properly. So we definitely leveraging AI because we did announce some feature called the Agentic Control Plane in September. Okay. That is focused on about exposing our context through the MCP protocol, but also providing the governance layer about what AI do and cannot do. I think when the fleet comes in, it’s more about we want to expose our data through the MCP protocol for any harness. They can leverage our data to do more sophisticated reasoning and control, but also providing a very strong governance layer for the AI to take actions through us. So there.

Swapnil Bhartiya: Can you also talk about how is it also different in updating across environments versus within fleets? The scale, the complexity, the challenge.

Hong Wang: Yeah, I mean, the environment in our concept is more about the type. So we are talking about dev, staging, production. So in the dev, you may have 5 dev environments. In staging, you may have another 6 or 10. But in our reality, when we look at our enterprise customers, their production environment is 500 to 1,000. It’s not uncommon. That’s where we’re thinking the Environment management is actually in every environment, we don’t even

Swapnil Bhartiya: know the location, everything.

Hong Wang: Right.

Swapnil Bhartiya: How does it also verify its target? And what if some rollout have to pull back? Oh

Hong Wang: yeah, that’s a great question. I think that’s also one of the core functionality of the Kargo. So we have this concept called the verification. So what it’s doing is, take an example, you deploy some functionality or new binary into a dev environment and it’s running, but it’s running is not good enough for you to consider to put it into the staging or production. So what we are doing here is we are firing up the verification. The verification could be triggering the functional testing against that environment, or could be querying the observability solution for the golden signal, kind of like certify, oh, this artifact in dev is good. So it’s certified for the next stage to pick up, normally staging. Staging also have its own verification, could be more sophisticated, could be doing more like stress testing, then it will be certified and then go to the next stage. Basically, Kargo already have those like a gate concept to be built in into the promotion. So basically our mission here is like we are providing like a continuous promotion with also like governance control, means it’s kind of like more automation. So it doesn’t matter the scale. You have 5 environment, you have 500 environment or 500 like target, it can always manage the bots.

Swapnil Bhartiya: Initially, you know, when we started, you gave a great example of Meta. What are the other use cases that you have seen where you’re like, You know what, we were not even envisioning it being used at that scale, and that use case made you wonder, oh, this is what they’re doing. So what are the use cases you can talk about that surprised even you?

Hong Wang: I think surprised or not surprised, it’s more like I’m starting to kind of see more and more in action. So let’s still go back to the Meta case. So Meta is, they have 100 clusters, they need to babysit the clusters well. Their original solution is using Terraform for everything. But they said is for a single line of the change of the configuration, it takes 4 hours to roll out. They figure out that doesn’t make sense. And then they realized, okay, for the core functionality of the infrastructure, they’re using Terraform, still working. But for the other system application, other business application, using Terraform is overkill. That’s how I see— they found that Argo and Kargo is a great combination to solving their problem. Mm-hmm. So they were able to decrease from 4 hours to several minutes. to do the deployment. Especially in AI, time is money. And the cluster is very expensive with all the GPU running there. So that’s one use case, basically AI training infrastructure, many clusters. Another one is what we found out is we have a lot of the retail customers. So they basically started running the Kubernetes in an individual franchise store. And the cluster is very expensive with all the GPU running there. Right. So that’s one use case, basically AI training infrastructure, many clusters. Another one is what we found out is we have a lot of the retail customers. So they basically started running the Kubernetes in an individual franchise store. So we are talking about the 1,000-store, 2,000-store, 3,000-store. And they are basically all production. So basically they need a kind of like, oh, this is my latest software, I need to put that into 1,000 stores across the United States or even globally. How you do that? That’s kind of like a very typical fleet management problem there. And the third one is also a unique one. I think as we are seeing more and more SaaS or traditional business and people and customers is asking for security boundary or even globally, how you do that? That’s kind of like a very typical fleet management problem there. And the third one is also a unique one. I think as we are seeing more and more SaaS or traditional business and people and customers is asking for security boundary, more isolation, So there’s a lot of the new idea about like bring your own cloud or maybe the per-cluster, per-customer architecture. So it means like as a vendor, when you have 100 customers, you may have 100 dedicated clusters for them. So it’s also another fleet problem to solve. So basically that use case keep coming up. It’s just like the infrastructure just keep growing.

Swapnil Bhartiya: Right. And since we are talking infrastructure and we are talking AI, I also wanna talk about, of course, Neocloud and AI infrastructure providers are adding clusters faster than any other enterprise that we have seen in past. What does fleet management look like at that pace, and what can other platform teams who are not even working at that scale can learn from it, and how are you actually tackling that space?

Hong Wang: So publicly, CoreWeave is our customer, and they definitely have many, many— as the neocloud, number one neocloud, basically they are leading the charge. And they basically has so many cluster and so many back office like infrastructure managed by them. So we are very proud that we’re working with them for the last like 3, 4 years. So we kind of like powering and managing their back office fleet already. That’s how we learned a lot and a lot from them. So overall is what we seeing the trend here is we are working with many other new labs or neoclouds there. One trend here is the GPU resources actually still scarce. And it’s actually also very scattered. Means that to be able to get enough GPU for very meaningful work, I think a lot of the big labs, they are still getting the GPU cluster from the different cloud provider. So some is from the big 4, big 2, big 3, sorry, AWS or Google or Azure. Some is from the neocloud. That means you always naturally, they have more use case for the multicloud, multi-region use case. That means they need a more out-of-box easy solution, be able to managing those infrastructure from the different location very easily. So it’s like the edge, but it’s the edge from the different cloud provider with many clusters to be scattered. And we actually designed our whole solution end-to-end for that use case very well because our architecture is a distributed agent architecture. So no matter where the cluster is running, even on your laptop, can be easily managed by our Akuity platform.

Swapnil Bhartiya: And as you know, of course, you gave all these examples. Actually, I was also thinking about if you look at hardware, The scarcity is there. So just

Hong Wang: check what’s the per GPU per hour price from Amazon as an example. It could be at $20 or even more. So our point is we give the people a very strong automation tool, can manage your infrastructure in the most effective way, can deliver your software in the most fast and reliable way. That’s the money saving for you. So that’s kind of our positioning.

Swapnil Bhartiya: Perfect. I also wanna talk about, you know, it’s going back to the point that you made earlier, you know, once a new cluster comes online, how do teams keep it from adding more manual orchestration. So the things are automated and, you know, the manual— because that leads to more complexity challenges.

Hong Wang: Exactly. That’s why when you have a 3, 5 cluster, you’re getting the 6th one, it’s fine. But nowadays we are talking about you’re adding another 10, 20. That’s where our automation actually working. Basically we have this like fleet concept, we have this kind of manual, because that leads to more complexity challenges. Exactly. That’s when you have a 3, 5 cluster, you’re getting the 6th one, it’s fine. But nowadays we are talking about you’re adding another 10, 20. That’s where our automation actually working. Basically we have this like fleet concept, we have this kind of target set concept, means it’s very easy. You get a new cluster, you label the cluster in the right way, it will join the fleet. It will join the template, join all the what needed to be deployed on cluster in a fleet-level template. Means it can like in no time it will be provisioned and managed in that way. So, and also the fleet is come and go because sometimes you’re getting a bigger one, smaller one. That elastic is also supported by us very easily.

Swapnil Bhartiya: Initially we started talking about your previous survey, you know, that you talked about. Oh yeah. What is the next survey you folks are working on to get the pulse? Because things change so fast in this time, you know, within not even months, within weeks, things change. So as to know where the market is heading, so what is next you’re planning to get a pulse on? Is it plans for the next year? Because it’s almost the end of the year.

Hong Wang: That’s a great, great question. I think the KubeCon is a great location. We learn a lot because we have some early understanding about what is the future of the infrastructure, what kind of like a new workload is running on the infrastructure. I definitely think the AI agent is the next big thing. So a lot of the new things that have been talked here, we are talking about the OpenShift, we are talking about the agent substrate, a lot of new projects emerging. So that’s kind of like where we are looking. I think that’s where we’ll spend more time down the road to understand it and building a customized solution there. Things are moving so fast now, like originally thinking about, oh, 3 years someone will be adopted. Nowadays it’s 6 months.

Swapnil Bhartiya: And since you mentioned the AI agent and we did touch upon that, but can you also talk about where do they help? with promotion and where should human still stay in loop or making decisions?

Hong Wang: I think AI agent definitely will be next stage about infrastructure management. I think definitely it’s a little bit slow for the good reason because we are managing the production environment. So means any malintentioned behavior means instant, means we are losing the revenue and everything. That’s why there are a little bit hesitancy about infrastructure management. I think definitely it’s a little bit slow for the good reason because we are managing the production environment. So means any malintentioned behavior means instant, means we are losing the revenue and everything. That’s why there are a little bit hesitancy there. But with the sandbox, with the control, with the rich context, with the stronger model, I think that’s definitely starting changing. And more and more people starting to at least feel comfortable to giving all the read-only access to the production data. And they’re starting to getting more comfortable to letting the AI agent to be able to changing and modify the production system properly. That’s why there’s more like AI SRE like solution agent are showing up. I think that’s kind of like the trend we are going.

Swapnil Bhartiya: Are you already evaluating or working on like, as you said, read-only or beta?

Hong Wang: Oh yes. That’s in our agentic control plane solution we launched in September. We are already providing all the end-to-end solution for our people. Whoever is going to run the AI agent and want to having the rich context, we are the context provider. Who is the one who want to taking action against that infrastructure? We providing the governance layer for people to taking action there. So we providing the unified authentication, we providing also the governance control policy. Okay. and auditability for their day-to-day operations. So that kind of, we are ready for taking that traffic already. And a lot of customers are already using that.

Swapnil Bhartiya: Let’s fast forward to, let’s say, I mean, I don’t want to say 1 year because it’s too long. I don’t want to say next week because that’s too short. But let’s say, you know, when it goes into production and just think about, let’s, you know, so when an agent promotes a change, what should it be allowed to do? At the same time, should it inherit the permissions that you talked about and approvals for the person that is asking? How do you envision that the production agents will look like?

Hong Wang: Wow, that’s hard because there are so many new ideas come and go and people are still trying. I do think the AI should have the read-only access at the bare minimum to all the production system, no doubt. And I do think the AI agents should be able to promoting things for the future. I think there are so many new ideas come and go and people are still trying. I do think the AI should have the read-only access at the bare minimum to all the production system, no doubt. And I do think the AI agents should be able to promoting things for the future. I think the only thing I think the gap is about the deletion and the most sophisticated operation. For example, we are talking about auto-memory, we are talking about performance tuning. I think those we should allow the AI to start doing. But for other more scary operations, I feel we need to build more muscle around it. So that’s kind of my take right away. But I don’t know, 6 months later it could change again.

Swapnil Bhartiya: Here’s another interesting thing that when we talk about agents, of course it’s not that agents are not just helping with delivery. You actually have to also deliver those agents. Right. Right? So how do you do the AI agent delivery with OpenShift

Hong Wang: and Kargo? And we wrote a blog, and that’s kind of like our experimentation and a demo. And it was very well received by the community. I think I’m getting about 50,000 impressions on LinkedIn, about like 500 reactions. I definitely think it’s a great, great topic. Everyone’s thinking about that. So to us, it’s like the old days, we are talking about the bare metal machine, VMs, container, microservice, the serverless, and the AI agent will be kind of like new kind of like a breed of the workload will be running on the infrastructure. I think we are ready and we are exploring, definitely will be managing that new emerging Like agent now. And also another thing, trying to call back to our fleet management, because you are not going to running one agent. It’s more likely about 100, 1,000, or 10,000 agents. So how to manage lifecycle of those agents is another challenge. I think with the fleet, with our long story and a long, like I would say, like experience with the VM, with container, with others, I think we are ready for taking the AI agent as our new workload. So we manage both, yeah.

Swapnil Bhartiya: Yeah, and that’s where, you know, things are heading either way. We are almost at the end of the year, you know, almost there. This is the last KubeCon of the year as well. Of course, GTC is there, so many other events are planned. But if I ask, you know, from your perspective, what are your plans for the— first of all, we cannot plan, right? Things change so fast. But what’s on your roadmap next when it comes to Akuity or Argo or Kargo?

Hong Wang: Yeah, I think definitely we want to double down, triple down on the fleet management again. It’s just like, that’s the trend, the infrastructure is growing in that aggressive way. They just got donated to CNCF not long ago. I think there is a lot of innovation in this space and we’ll be watching them, we’ll be participating. And we are trying to actually incorporate all the knowledge into our solution properly. Means that’s the runtime level, basically how you run the agent, but how you deploy the agent, how you manage the lifecycle of the agent, it will be part of our story and journey down the road. So that will be very important for next year.

Swapnil Bhartiya: Hong, thank you so much for taking your time out and of course talk about Akuity, talk about Argo, Kargo. Thank you for your time. As usual, I look forward to chat with you again.

Hong Wang: Thank you. Thank you also for having me. Thank you.

Swapnil Bhartiya: Appreciate it. Thank you.

SIOS and NTELS Partner to Strengthen Enterprise High Availability in South Korea

Previous article