By AI token costs accumulate silently at the developer and team level, then surface weeks later as a shock to engineering leadership and the CIO. By the time corrective action is possible, the budget damage is already done. Meanwhile, agentic applications introduce non-deterministic behavior that classical observability tooling was never built to handle, creating a second, parallel visibility gap around quality, accuracy, and trust.
In this interview on TFiR, Raja Mukhopadhyay, VP of Product Management at Splunk Observability, covers the general availability of Splunk Tokenomics and the full Splunk Agent Observability solution, including runtime guardrails, AI evaluation frameworks powered by the Luna small language model, Observability Studio for instrumentation, and the multi-domain incident surface provided by Cisco Cloud Control.
Guest: Raja Mukhopadhyay, VP of Product Management at Splunk Observability
Show: TFiR
Here is what every platform engineer, SRE, and engineering leader needs to know.
Technical Deep Dive
Q: Why are AI token costs invisible to the teams generating them?
Raja Mukhopadhyay, VP of Product Management at Splunk Observability, explains that practitioners such as developers using coding agents are entirely disconnected from the cost side of their AI usage. Costs roll up to the CIO level on a monthly basis, creating sticker shock long after the spend has already occurred. There is no real-time signal at the point of use to help the individual modulate their behavior before budgets are exceeded.
“The thing is that today most people that are the actual practitioner, they’re so far removed from the cost side of it, the cost rolls up maybe perhaps a month later at a CIO level and then there is sticker shock.” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: How does the AI token cost problem compare to earlier technology adoption waves like public cloud?
Mukhopadhyay draws a direct parallel to public cloud adoption in the mid-2000s, when engineering teams deployed workloads without organizational alignment, giving rise to shadow IT. It took years before enterprises developed the governance, right-sizing practices, and architectural frameworks needed to manage cloud costs effectively. The AI token cost problem follows the same pattern: initial enthusiasm drives usage before governance structures exist to contain it.
“When you look at big technology waves, there is a particular pattern. First, there’s a lot of enthusiasm and excitement about usage. We saw that in public cloud when it came about in the mid-2000s.” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: What two core problems does Splunk Tokenomics solve?
Mukhopadhyay identifies two distinct problems Tokenomics addresses. The first is real-time cost visibility so that practitioners can self-correct before budgets are breached. The second is tying token spend to business outcomes, so that organizations can answer not just whether they stayed within budget, but what value the AI investment actually produced.
“The first thing is to get that visibility and then the second thing is to make sure that it’s not just about the cost, not just about the visibility, but also to have metrics that tie into what is the AI doing for the business.” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: How does Splunk Tokenomics deliver real-time token cost visibility at the developer level?
At the developer level, Tokenomics surfaces per-session cost in real time so that a developer using a coding agent can see the running cost of their current session and decide whether the work justifies continued spend. Mukhopadhyay gives the example of a developer seeing their session approaching $30,000 in token spend and being able to step back, reconsider their approach, and choose a different course of action before the cost compounds further.
“If I look at my token spend creeping up and I’m already past 30,000, I know that I need to step back and think it through again, perhaps decide on a separate course of action.” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: How does Splunk Tokenomics serve engineering leaders and CIOs, not just individual developers?
Tokenomics aggregates cost data at multiple organizational altitudes. A VP of Engineering running a thousand-person team receives real-time rollups so they can see by month one whether they are on track to exhaust their annual token budget and take corrective action early. At the CIO level, the solution provides granular visibility sliced across business units and functional teams, covering engineering, sales, marketing, and other groups, so AI spend can be governed like any other enterprise technology budget.
“With our tokenomics solution that VP of engineering will get real time updates that across the organization rolled up month one, this has been the spend so that they can go like, look, if in month one I’ve already started to spend like a quarter of my annual budget, I need to ratchet down.” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: How does Tokenomics connect token spend to business value, not just cost?
Beyond budget compliance, Tokenomics introduces value metrics that let teams quantify what their AI spend produced. For an engineering team, this means metrics like lines of code generated or development cycles compressed from months to weeks. Mukhopadhyay is explicit that staying within budget is a prerequisite, not the goal, and that the more important question is what the business actually got out of the spend.
“Someone’s got to be asking the question, all that spent, what have we gotten out of it? In many ways that’s the more important question, like what is the business getting out of it?” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: Why does agentic AI require a fundamentally different observability approach than classical microservices?
Classical applications are deterministic: developers know what logic they have encoded, bugs can be debugged, and behavior is predictable. Agentic applications shift the human-software interface to the level of intent expressed in natural language, making the underlying engine non-deterministic. This means observability can no longer simply ask whether a service is slow or a Kubernetes cluster is healthy. It must now answer whether the application is actually doing what the developer intended it to do.
“Applications inherently have now become non deterministic and therefore that has huge implications on observability. Because if you look at the classical frame of observability, it’s all about is my service running slow? And now that has shifted to can I really trust that my application is doing what I want it to do?” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: What does the full AI observability stack look like in Splunk Agent Observability?
Mukhopadhyay describes the agentic layer as an intelligence layer sitting on top of classical infrastructure, not a replacement for it. Agentic applications still run on microservices, Kubernetes clusters, and databases. Splunk Agent Observability is designed to provide visibility into both the classical infrastructure stack and the agentic layer simultaneously, enabling developers and SREs to quickly isolate whether a problem originates in the agent logic, the service layer, or the underlying infrastructure.
“We need solutions like Splunk Agent Observability that provides visibility into both the classical stack and the agentic layer, and more importantly allows the developers to swivel very quickly and isolate, like, hey, is this problem because of something wrong in the agent, or is this problem a classical problem in the service layer?” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: What out-of-box and custom metrics does Splunk Agent Observability provide for agentic applications?
Splunk Agent Observability ships with out-of-box metrics that work across all agentic applications, covering whether responses are being returned, whether they are accurate at an aggregate level, hallucination rates, and the sentiment and tonality of responses. Custom metrics allow individual development teams to define additional signals that are specific to their application’s behavior and business context.
“Out of box metrics are kind of like that work across all applications. Custom metrics allows developers to specify metrics that are relevant to their particular application.” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: What are runtime guardrails in Splunk Agent Observability and why are they necessary?
Agents configured to take actions on behalf of users can, without proper constraints, execute actions that are damaging or catastrophic to the business. Splunk Agent Observability provides a policy framework through which developers can define constraints on agent behavior. These policies are enforced at runtime, actively checking and limiting what the agent can do as it operates in production.
“Unless they are constrained properly, sometimes they can take actions that depending on the context can be catastrophic or can be hugely damaging to the business. So our agent observability solution provides developers a framework through which they can specify policies that would constrain and in runtime actually check the behavior of the agents.” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: How does Splunk Agent Observability approach AI evaluation and how is it different from classical software testing?
Classical software testing relies on unit tests and system tests against a predefined set of scenarios. Agentic applications generate a continuous stream of prompts and responses in production that collectively define a far richer universe of real-world test cases. Splunk Agent Observability harvests this data to build evaluation data sets that development teams can run against new versions of their agents, ensuring that changes to the agent harness or encoding logic do not degrade response quality for known prompt patterns.
“The prompts going into the model and the responses coming back in their totality define sort of a universe of test cases that if harvested properly can be used as look, anytime the developer team is making changes to the agent harness to the encoding logic, you run the eval data set and just make sure that given a particular kind of prompt that you get a correct response.” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: Why does Splunk use a purpose-built small language model called Luna for AI evaluations instead of frontier models?
Running evaluations at scale using frontier models reintroduces the same token cost problem that Tokenomics is designed to solve. Splunk invested in building Luna, a specialized small language model, to power evaluations in a way that is both cost-efficient and extremely low-latency. The model is purpose-built for evaluation tasks rather than general-purpose reasoning, which makes it well-suited to the volume and speed requirements of production-scale AI quality checks.
“At scale, if you use the frontier models for evaluations, again going back to the cost, it’s in most businesses that the cost math is going to be off. So that is why we have built and invested in a small language model that is purpose built for doing evaluations at scale.” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: What is Observability Studio and how does it help developers instrument applications correctly from the start?
A persistent problem in observability is that developers only discover missing telemetry after an incident, when they need data that was never captured. Observability Studio addresses this by embedding a set of skills into coding agents such as Claude Code or Cursor, allowing developers to instrument their applications correctly using natural language prompts from within their preferred IDE. The goal is to ensure the right telemetry exists before problems occur, rather than discovering gaps during troubleshooting.
“Through these skills you can go and using sort of very easy natural language prompt means you can instrument your application the right way.” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: What is the full AI software development lifecycle that Splunk is trying to automate?
Mukhopadhyay describes a full cycle that Splunk refers to as AISRE, spanning instrumentation, problem detection, troubleshooting, and remediation. Remediation can include changes at the infrastructure level or within application code. The objective is to remove manual toil from each stage of this cycle so that engineering teams spend less time reacting and more time building.
“We are taking the toil through our AISRE efforts all the way from instrumenting to detection of problems to troubleshooting and then finally with remediation, whether the remediation requires some change in the infrastructure or in the application code.” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: How does Cisco Cloud Control unify observability across security, network, and engineering teams?
In most enterprises, security operations, network operations, IT operations, and engineering teams each use separate, domain-specific tooling. When an outage occurs, its root domain is unknown upfront, so all teams investigate simultaneously, resulting in significant back-and-forth before anyone isolates the actual source. Cisco Cloud Control provides a single AI-powered, multi-domain surface where all of these teams can work from shared context and vocabulary, dramatically accelerating the process of isolating whether an incident is a network issue, an application problem, or a security event.
“It is the single AI powered surface that is multi domain. And that multi domain part is huge because it allows for the first time the security team, the IT operations team, the engineering team, the networking team to come with a shared context and a shared vocabulary to reason about an outage and to be able to isolate it into the proper domain very quickly.” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: What self-reinforcing loop does Splunk envision between AI observability and agent quality improvement?
Mukhopadhyay describes a future state where AI observability does not just monitor agents but actively feeds correction signals back to them in real time. When an agent takes an incorrect action, the observability layer catches it, immediately feeds that signal back to the agent, and the agent adjusts its behavior. Over time, this creates a self-reinforcing loop that continuously improves the precision and accuracy of agents running in production without requiring manual developer intervention for each failure case.
“You can see when we get to that point, that sort of self reinforcing loop that would present like a real unlock in driving kind of quickly the precision and accuracy that customer can expect out of their agents.” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: How does AI observability help engineering leaders maintain accountability even when the underlying engine is non-deterministic?
Mukhopadhyay is direct that non-determinism in the AI engine does not reduce the accountability engineering leaders face for application behavior. Customers and business stakeholders expect applications to work correctly regardless of what technology powers them. Splunk’s AI observability work is focused on giving developers and engineering leaders the same effective control and confidence they had with classical deterministic applications, even though the engine beneath has fundamentally changed.
“The expectation and the accountability that you will be held to is still a deterministic one. Your application needs to work all the while while the underlying engine has been switched overnight into a non-deterministic one.” — Raja Mukhopadhyay, VP of Product Management, Splunk
Q: How does Splunk see the AI software development lifecycle shifting left?
Mukhopadhyay describes a shift where observability data and visibility are brought forward into the development lifecycle rather than being applied only after deployment. This means engineering teams can use observability tools to answer the question of whether a new agent version is ready to ship to customers, using real production data from prior versions to validate the change before release.
“How do we help what we call the AI software development life cycle, because this is now left shifting, like taking the data and the visibility that observability tools have and bringing it up front into the development life cycle so that as developer teams are making changes, someone has to answer the question, is this new version ready to be shipped to customers?” — Raja Mukhopadhyay, VP of Product Management, Splunk
Resources & Documentation
- Splunk Observability Cloud, full-stack observability platform covering infrastructure, APM, and agentic AI layers
- Splunk, machine data platform providing operational intelligence for security, IT, and DevOps
- Cisco Cloud Control, multi-domain AI-powered operations surface for network, security, and IT teams
***
👇 Click to Read Full Raw Transcript
Swapnil Bhartiya: Hi, this is your Sapnin Bharatiya and we are here [email protected] and today we have with us Raja Mukopatiya, VP of Product Management at Splunk Observability. Raja, you know, these days every enterprise, they are racing to deploy AI and it is exciting also sometime they get carried away. They build so many agents, they put everything in the AI, but it happens only very late when they realize the cost problem, and that is tokens. Because in most cases spends stay invisible until you get that invoice. And by that time it’s only too late. But organizations are trying to address this problem. Tokenomics has become one of the most heated topic these days. AGI. AGI actually secondary to economics is primary because that’s how this will be sustainable. So before we talk about what Splunk is doing, what you announced here at the conference, let’s talk about this problem area. How do you see it?
Raja Mukhopadhyay: It is a very real problem, right? And the way I would think about it, in some ways it is common. When you look at big technology waves, there is a particular pattern. First, there’s a lot of enthusiasm and excitement about usage. We saw that in public cloud when it came about in the mid-2000s. I mean, there were engineering teams that went ahead and deployed workloads on the public cloud, even though the larger organization was still not there. And that led to the rise of shadow IT and everything. And then it took several years for people to realize the true value of the public cloud, but also wrap around the right governance so that the cost and which workloads to put in the public cloud and kind of like how to right size workloads, the right architecture, all these things evolve. Now the same pattern we see with using AI and to your point about the cost of tokens and all that, right now we are in the phase where there’s a lot of genuine excitement and enthusiasm about the use of AI. So whether it is developers who are using the coding agents and building agentic applications, whether it is the use of other kind of functional roles in the enterprise that are using tools like Claude Cork and such to get productivity gains, there’s a lot of usage of AI. But to your point, the thing is that now we are starting to see that in the end someone has to pay the bill. And a lot of enterprises are recognizing that they’re struggling really to reconcile the spend on AI with the value that they’re getting out of it. And it comes down to kind of like the same principles, if you will, is the first thing to your point, why is this a problem? Is there is not enough visibility? Like, if me, when I’m a developer and let’s say I have a session with a coding agent and I spend the last hour like developing a piece of code, if I have visible visibility into like, look, right now I’m using and this thing is costing me, you know, whether it’s, you know, $100 or $500, that would give me a sense of like, look, the work that I’m doing, Right. Is this really worth that? Now the problem is that today most people that are the actual practitioner, they’re so far removed from the cost side of it, the cost rolls up maybe perhaps a month later at a CIO level and then there is sticker shock. So the first thing is to get that visibility and then the second thing that we, and we get into this in a bit is also to make sure that it’s not just about the cost, not just about the visibility, but also to have metrics that tie into what is the AI doing for the business. So tying the token spend to what outcomes have been driven for the business.
Swapnil Bhartiya: Right.
Raja Mukhopadhyay: And so you, you know, fully agree that this is perhaps right now, at least in this particular point in time, the most sort of challenging problem that a lot of enterprises have. And that’s why we can talk about what Splunk and Cisco, we are doing to help our customers in that when
Swapnil Bhartiya: they look at tokenomics, how does it give teams kind of real time visibility into token spend so that you do know where you are? Also, as the context pile up, the token calls also stops doubling up. It’s not as simple. So whether you are using autonomous agents or whether you are using tools like cursor or cloud code. So talk about how tokenomics actually tries to address this problem.
Raja Mukhopadhyay: Yeah, and I’ll talk about what we are doing. Right. So you heard in the keynote. So we announced the general availability of tokenomics, which is part of the Splunk agent observability solution. And really the motivation for that is at multiple levels in the organization. What I mean by that is like, to your point, if I’m a developer and I’m using a coding agent and I’m just at it for a couple of hours developing a bunch of code right there, using our tokenomic solution, you’d be able to find out that, look, this session it cost me $500. Right. So the real thing is real time cost visibility that would allow the practitioner to modulate their actions in real time so that they go like, look, this thing that I’m doing that is not worth $50,000. So if I look at my token spend creeping up and I’m already past 30,000, I know that I need to step back and think it through again, perhaps decide on a separate course of action. So that’s one. Now the other thing with our tokenomic solution is that that visibility and the aggregation of the cost data is available at multiple levels of the organization. So me as a developer, of course, you know, I go and all I’m doing is like, I have an intrinsic view of like look, the work that I’m doing, you know, it clearly is not worth like 50,000 or $100,000. So that visibility is a check on me. Now if I go one step up and let’s say there’s a VP of engineering and they’re running a thousand person engineering team right now, of course for them they need to know like, look, I have a budget of say several million dollars that I can afford to spend on tokens for the year. Right? And it is important to what you just said a bit earlier that right now, the thing is oftentimes they get that visibility way later when the problems already occurred and then they’re trying to dig themselves out of the hole. With our tokenomics solution that VP of engineering will get like real time updates that across the organization rolled up month one, this has been the spend so that they can go like, look, if in month one I’ve already started to spend like a quarter of my annual budget, I, I need to ratchet down, I need to kind of like trim the sales back and then you go one level up to the CIO of an organization, right? I mean there the CEO has, let’s say the company has decided that they have a particular budget for AI and the CIO has been charted with making sure that across the different business units, the different function that spend is allocated well and that the AI, the different teams, they stay within the AI budget much like any other technology spend within a company. Right? And with our tokenomic solutions, we will give the CIO that granular visibility that’s rolled up across the different cuts that are relevant for them. So you are trying to look across the different business unit, you are trying to look across the different function. How much is the engineering team spending, how much is the sales team spending, how much is the marketing, all those different operational cuts that are there rolled up. So that’s one big piece of what we are doing. The second big piece that we are doing is with our tokenomics solution we will bring it. The question of course is
Swapnil Bhartiya: the first
Raja Mukhopadhyay: problem is, hey, let’s stay within our budget. That’s okay. And if you haven’t done that, then almost nothing else matters. So you have to stay within budget. But once you’ve stayed within budget, someone’s got to be asking the question, all that spent, what have we gotten out of it?
Swapnil Bhartiya: Right.
Raja Mukhopadhyay: And in many ways that’s the more important question, like what is the business getting out of it? Right. So with our tokenomics, you will see ways by which we are going to let customers get a handle on what is that token spend leading to in the context of their business. So for an engineering team, for instance, that’s developing code, it would be things like, look, we are generating these many lines of code, something that would have taken us say six months to develop. Now we can develop in a matter of, say, a month. So a focus on value metrics in addition to the cost visibility that tokenomics
Swapnil Bhartiya: presents, because cost control is never going to be like, you cannot grow a business based on cost control. It should be the value that you’re getting. Now the interesting thing, as I was saying earlier also that agents don’t behave like traditional microservices or kubernetes workloads. They are non deterministic. So what does. If we just switch and look at the tech stack, what does the full stack AI object look like when you have to also take into account the hallucination, latency spikes, data leakage, that is part of the real world as well.
Raja Mukhopadhyay: Yeah, this is a profound shift. Like if you look up to this point, applications have been deterministic. So as a developer, I know what I’m coding. Sure, my code may have bugs, what I have in my head may be different than what I have encoded as logic in my code. But bugs can we troubleshoot, we debug. But I have a general idea that my application is actually going to be doing this. Now with AI, the thing is the interface at which human beings are interacting with software that has gone up to the level of intent. So if you look at what we are calling an agent or an agentic application, if you deconstruct it, at its core, it’s effectively a developer saying, look, I want the application to do this in natural language. Now there are modalities of making it more precise and accurate. And that’s why the harness and all of the prompt engineering and things like that. But if you simplify it, it’s effectively a human being describing what I want my application to do. And now the question is, is the application actually doing what I want it to do? And that is the big shift where applications inherently have now become non deterministic and therefore what does it mean to that has huge implications on observability. Because if you look at the classical frame of observability, it’s all about is my service running slow? Is something wrong in my Kubernetes cluster? And now that has shifted to can I really trust that my application is doing what I want it to do? And therefore, like, if you look at our agent observability solution, the first thing is we need to have a solution that covers the full AI stack. What I mean by that is there clearly is the agentic layer, but these agentic applications, they’re not running on Ether, they’re running on microservices, they’re running on Kubernetes, they’re still making database calls. All that traditional stuff does not go away. In fact, if anything, the agentic layer is a layer on top. It’s an intelligence layer on top of the classical stuff. So we need solutions like Splunk agent observability that provides visibility into both the classical stack and the agentic layer, and more importantly allows the developers to swivel very quickly and isolate, like, hey, is this problem because of something wrong in the agent, or is this problem a classical problem in the service layer? Is this problem something that has to do with the infrastructure on Kubernetes or the database? That’s one piece. Now to your question. If I go and unpack our solution at the agentic layer, there a few things. If you look at Splunk agent observability, we provide out of box and custom metrics effectively. These are ways to tell developers and SREs can figure out if the agentic application is running in production. The responses are coming. Are the responses accurate or not at an aggregate level, how many of the responses are just the model hallucinating? Things like that? What is the sentiment, the tonality of the responses? Is it exactly what I want it to be? Right? We have custom metrics. Out of box metrics are kind of like that work across all applications. Custom metrics allows developers to specify metrics that are relevant to their particular application, right? The second big thing that we have is the notion of runtime guardrails, right? So what you saw in the keynote demo, like agents, if you want them that way, they can take action on your behalf. And unless they are constrained properly, I mean, sometimes they can take actions, actions that depending on the context can be catastrophic or can be hugely damaging to the business. So our agent observability solution provides developers a framework through which they can specify policies that would constrain and in runtime actually check the behavior of the agents. So that’s a big piece of what we do, right? And the third thing that I will highlight is the notion of doing evaluations. And this is a net new thing that has come up in the notion of agentic application. What I mean by that is if you look at a classical software application testing was what was the modality? Developers, they do unit testing on the code they write and there’s a team, an engineering team that does system testing. Is the system overall acting? And what was system testing? I’ll do a bunch of tests and I convince myself that look, it mostly passes these tests so it will do what we want it to do, I think. Now the thing is, when you look at these agentic applications, observability products effectively are sitting on a gold mine of data because the prompts going into the model and the responses coming back in their totality define sort of a universe of test cases that if harvested properly can be used as like look, anytime the developer team is making changes to the agent harness to the encoding logic, you run the eval data set and just make sure that given a particular kind of prompt that you get a correct response. So that’s a third big piece that makes our agent observability solution powerful. And in that we have some customers, you heard from Verizon in the keynote that uses our agent observability solution and they use, we have a specialized small language model, the Luna model to power our evaluations. And this is important because at scale, if you use the frontier models for evaluations, again going back to the cost, it’s in most businesses that the cost math is going to be off. So that is why we have built and invested in a small language model that is purpose built for doing evaluations at scale. Evaluations that are cost efficient and are extremely low latency.
Swapnil Bhartiya: One thing with AI is that everybody is offset with bigger model, but in most cases actually smaller models, they’re much more effective, not only in terms of cost, but they’re more focused on just solving that one problem. Now if you look at you folks are also extending agent observability across Splunk Observability cloud and of course Cisco Cloud Control. Talk a bit about how does that help developers build of course application that are born observable from day one, not as an afterthought of Course, you folks have done a lot of work in observability space, already so obsolete. I mean, we have seen a lot of paradigm shift, you know, a lot of things that used to be an afterthought. Some of this problem have been started baked in day one. And also talk about how does unifying Splunk’s machine data with Cisco’s network context give teams capabilities that they did not have before. Interesting thing with Cisco is that it sits at the center because that’s where the network everything goes. So you are in a position that a lot of people want to be. So talk about this aspect.
Raja Mukhopadhyay: Yeah, so maybe let me start with the kind of the developer thing first with Observability Studio there. Our motivation is, if you look at the practice of observability today, it still remains a fairly complex exercise. What I mean by that is, yeah, there are observability products and solutions, but the burden, the cognitive burden still squarely lies with the human developer and SRE team. What I mean by that is one of the big things, like let’s say you have an application that the developer team has developed. It goes into, I don’t know, a few million lines of code. Now, observability products, whatever telemetry comes out of the code, they give you a means to process it and then display the data, all that. But the real question is, who decides what is the appropriate telemetry data that needs to come out of the code? Right now the reality in most teams is like, and way back when I was a developer, I can relate to this. You go in and you troubleshoot an issue and that’s when you realize, oh, it would have been really nice if I had this additional piece of data so that I can actually see what the state in my application was. But I had not instrumented my application, so I don’t have it. This remains a pervasive problem because there’s no one helping developers figure out, look, if this is your application, this is the right telemetry data that you should have so that when something breaks, you have the full picture, the full state, and you would be able to troubleshoot effectively. Right? So that is what we are trying to do with the Observability Studio is we have a set of skills that we effectively pass into the coding agent. So whether you are working on cloud code, code edge or whichever IDE that you prefer, through these skills you can go and using sort of very easy natural language prompt means you can instrument your application the right way. So that’s a big Piece of that. And that is a part of the broader thread where we are taking the toil through our AISRE efforts all the way from instrumenting to, to detection of problems to like troubleshooting and then finally with remediation, whether the remediation requires some change in the infrastructure or in the application code. Right. So that full cycle. Right. I think the other thing that you brought about is, you know, with Cisco Cloud Control, right. So I think one of the things that you see in a lot of organizations, let’s say there is a observability solution and the engineering teams, the SRE teams, the IT operations team, they’re using that. The security operations team, they have a different SIEM security solution that they are going into. Oftentimes the network operations teams, they have specialized solutions that they look into. Now the nature of the problem is that when there is an outage, the outage doesn’t come with a handy ticket. That this outage is because of like a kubernetes cluster being stuck or there is a denial of service attack going on or some agent rogue that is hacking into your system or there is a problem in some network. Now in most enterprises what happens is because that problem, a prior is not known which domain is the source, is it the network, is it the application, is it like a security attack? Because that is not known. All these teams independently get fired up till someone goes like look, I think it’s the network or I think it’s the application. Lot of back and forth. And that is what we are trying to solve with Cisco’s cloud control. It is the single AI powered surface that is multi domain. And that multi domain part is huge because it allows for the first time the security team, the IT operations team, the engineering team, the networking team to come with a shared context and a shared vocabulary to reason about an outage and to be able to isolate it into the proper domain very quickly. So lots of exciting things that we are doing on that front. And in many ways it’s kind of the tip of the iceberg because I think that these are with using AI as the underlying technology block. Some of these problems that were effectively restricted to the particular domains, we are now able to bring in experience surfaces that span across these disparate domains and, and allow different teams to quickly get to the resolution. Like look, this problem is a network problem and the right agents get invoked and the problem gets resolved. Or this problem is kind of an application stack problem and the right fixes or this problem is a hack and let the security Agent come in and take the right quarantine action, so on and so forth.
Swapnil Bhartiya: Can you share any example if Verizon was enough, Dennis Fine, where tokenomics was able to kind of catch code before it became problem. And second part is that how do you also see the whole AI observability to economics evolve as of course agent AI is going to become a critical piece of how we write and deploy software.
Raja Mukhopadhyay: I think one of the things in general what we are seeing from I would say most of our customers, if I have to characterize where they are in the journey, just because the space is so new, they’re mostly right now the crime need today is visibility at different levels. So we have tons of examples of customers where that visibility has allowed different teams at different altitudes, all the way from individual developers to engineering leaders to the CIOs to modulate, to modulate and take corrective action to bring their AI spend in line with what the business can afford.
Swapnil Bhartiya: Right.
Raja Mukhopadhyay: I think to your second pat going to the question how agentic AI is going to be evolving? I think really it’s going to proceed across several vectors. So the first one is if you
Swapnil Bhartiya: look into not how AI agentic, but how AI observability and tokenomics will evolve as agentic AI is becoming more adversaries.
Raja Mukhopadhyay: So for AI observability, I think the real thing that we are focused on is to make sure that developers and engineering teams have full visibility, kind of get them to the same deterministic frame that they had with classical applications, even though the underlying engine has become non deterministic. What I mean by that is if you are an engineering leader, you don’t get a pass by saying, look, my application is doing something incorrectly. That’s because it’s an agent application. No one is going to give you that pass. So the expectation and the accountability that you will be held to is still a deterministic one. Your application needs to work all the while while the underlying engine has been switched overnight into a non-deterministic one. And that is the key thing that we are trying to solve with AI observability. How do we make it easy for our developers, for SREs and the leaders to get that? That’s one. I think the second big thing that we are focused on is how do we help what we call the AI software development life cycle, because this is now left shifting, like taking the data and the visibility that observability tools have and bringing it up front into the development life cycle so that as developer teams are making changes and someone has to answer the question, is this new version ready to be shipped to customers? How do you do that? Observability products have a unique opportunity to help engineering teams answering and wrestling with that question. Then a little further out, and this is the thing that we think is most exciting is this notion of using AI observability to improve the quality of our customers agents. If you look effectively, the mental model is that of a self reinforcing loop. You have an agent, we have our AI observability solution. And the agent does something and it takes some incorrect action and the agent, AI observability catches that incorrect action, feeds it back, feeds it back right then and there so that the agent knows this was incorrect and then it makes a change. And you can see when we get to that point, that sort of self reinforcing loop that would present like a real unlock in driving kind of quickly the precision and accuracy that customer can expect out of their agents.
Swapnil Bhartiya: Raja, thank you so much for joining us. And what I really like is that the focus is less on cost control, but actually what you get out of the investment you made on AI. Once again, thank you for your time today and I look forward to chat with you again.
Raja Mukhopadhyay: Absolutely. Pleasure talking to you. Okay, thank you.
