Why AI Agents Get Enterprise Data Wrong | Raviv Levi, CData | TFiR

Swapnil Bhartiya: As we all know that AI agents are moving very fast from answering questions to now taking actions. They are booking things, updating records, running workflows on their own without any intervention. But here’s the catch. Most agents don’t actually understand your business context. They don’t know your system, they don’t know about your data and they don’t know your context. So, so what happens when they act? They are largely guessing. And a guest that touches a customer record or financial system isn’t just wrong, it is risky. Enterprises want the speed of agent, but they don’t want this exposure. That is the exact tension CDATA is stepping into with its new Connect AI Gateway to bring that company context and governance to AI agents and to dig deep. We have with us today Raviv Levy, Chief Product and technology officer at CData. Raviv, it’s great to have you on the show.

Raviv Levi: Thank you so much for having me. Swapnil really excited about Connect AI and the AI transformation all around us.

Swapnil Bhartiya: I am excited as well. Now we have been covering CData for a very long time, but no pun intended, the context is changing here. So I would love to know if you can explain from the perspective of this announcement that you folks are making today. We will talk about the announcement separately, so let’s just keep that as a teaser. Let’s talk about seadata, who you folks are, what do you do and what role do you play or you are going to play in enterprise data connectivity space where AI agents are becoming more and more pervasive.

Raviv Levi: In a nutshell, CDATA as a company has been around for over a decade working diligently to perfect data connectivity capabilities. Starting with connectors and drivers, moving up to data replication, data virtualization and so on has been used by over 10,000 customers to connect enterprise to their data sources. Whether SaaS, Cloud, on prem, you name it, CDATA has been the solution of choice for those kind of data connectivity needs. What’s happened recently, interesting enough, is that it’s no longer data engineering teams that require that level of connectivity, but AI. And through AI, practically every employee, human or agentic in the company now needs live access to data. Now how you do that and how you enable that and how you partner with those companies, with those customers in their AI transformation journey. This is what we’re about here at CData, transforming and evolving along with our customers and with the market from a place of pure and simple data connectivity to the more advanced needs that we see from AI applications, platforms and agents.

Swapnil Bhartiya: Excellent. Thank you for explaining that. And that Also sets the stage, the foundation, to better understand what you folks are announcing today.

Raviv Levi: Yeah, so as mentioned, the name of the game is AI Transformation. And I’ve been spending a lot of time with partners, analysts, customers, tracking down the industry progress and how we treat AI. There’s been a lot of innovation out there. On a weekly basis, you hear so many different announcements. It truly is an exciting time to be alive. But when you think about that in the business world in B2B, how are you able to, to really harness that power to improve your business, to show preferred results? Well, the answer to that is that sure, you have the intelligence, but you have to be able to connect that to data and you have to be able to do that in a responsible, efficient, accurate and secure way. And this is where CDATA and the Connect AI Gateway comes into picture. Connect AI Gateway is a solution that is built in order to help our customers transform their own organizations in order to become AI native themselves. It’s a high level speak to say that you need a component in your organizational architecture that would allow you to connect employees, humans and agents into models and, and data on the other side. And the way that you do that through an AI gateway would really determine the results that you’re seeing. There are many benefits to that. We’re going to talk about context in a bit, but even before we dive into context, just think about that. Right now you have employees and agents running around accessing all kinds of data sources, all kinds of models, no clear visibility to the organization of what’s going on, who’s paying for what, how much does it actually cost us, and a lot of security concerns as well. By architecturally pointing out an AI gateway and making all those requests funneled through it, you gain a lot of advantages for the enterprise. You get the observability benefits, you understand what’s going on and who’s using what. You have security and governance benefits. And you can also apply policies and cost optimization mechanisms that would help you in your AI transformation in a more efficient and responsible way.

Swapnil Bhartiya: Now, as we all know that companies are moving from AI stands that used to just answer questions to AI agents that actually take action on their own. What new challenges do they pose to enterprises that are running into, you know, when it comes to challenges related to data, context or governance?

Raviv Levi: Yeah, so it’s, it’s all about the context. As, as you correctly mentioned before, when we’re thinking about the ability of an AI entity to help us in our business, they have the intelligence, they have access to data, but they need guidance and they need guardrails and they need to be explained. When we ask for this question, what do we actually mean when I say enterprise revenue in my company’s context, what does that mean? Does that mean the revenue of the company as an enterprise? Does that mean the revenue that is coming from our enterprise customers? How do we even define our enterprise customers when I’m asking about the customers that are at risk for this quarter, how do we define customers that are at risk? When I’m looking for a specific piece of information, where is it stored? All that information and metadata is what comprises of the context, the enterprise context that helps AI tools and agents to really get the answer that you need. That is step one. Even without getting into agents that are taking action, you have to get the right answer. Interesting enough in comparisons with other tools that I shall not name right now, we’ve seen that just by being able to ask the right questions and put the right context in front of AI tools, the accuracy of the answers that we’re getting is jumping from 65% to to over 98% and that’s regardless of models that you’re using. So the cost reduction could be more than 100x because now you don’t necessarily need the most intelligent model out there to get a correct and accurate answer. Now let’s go to the action piece. That is the obvious next step. But in order for the action to be meaningful, you need two things. You need to know what you’re talking about. So that’s the accurate answer. And now you need to be able to access a variety of tools and data sources. And this is really one of the benefits that CDATA has. Some of the work that has been done over the last decade, if not more than that, there’s been hundreds and hundreds of data sources and connectors that are now being made available through the platform, in addition to natively supported MCP servers that different vendors would offer. But more interestingly, even if some vendors don’t have native MCP offerings, we would support that. So now your agents are able to research, get the right answers and take actions in the corresponding avenues of your choice. And just as a quick anecdote and example, I have an agent that is doing exactly that. So my work agent is connected to all my SaaS and cloud data sources, obviously through Connect, AI gets the right answers and is able to take action, whether it’s to set a meeting for me, update JIRA ticket, update a Salesforce ticket, create slide decks and so on completely autonomously. And nine out of ten times I would say fully accurate.

Swapnil Bhartiya: Now, as you rightly explained that the whole Connect AI gateway is built around the idea of bringing enterprise context to agent. Can you talk about context may mean different thing in different context altogether. What does context mean here and how does gateway build and keep improving that context over time? Because company data continues to evolve as well.

Raviv Levi: Yes, that is a great question. We talked about multiple types of context so far. There’s the metadata that is coming from the data sources. There are some types of contexts that could be imported into the gateway, whether coming from your data store, like snowflake databricks or others. But there’s another type of context that is really, really important. There’s a lot of conversation in the industry these days around a concept that’s called tacit knowledge or institutional knowledge. This is the kind of thing that employees, new employees joining your company would have to learn. That’s something that’s not really written anywhere. Someone would need to teach them. That creates a problem for agents, as you can imagine, because there’s no one there by the agent to teach the agent about those things that are not written anywhere and only employees would know one. One example could be how do I open a ticket? How do I write down a ticket for the engineering team? There are certain nuances or certain things that would go there, or who do I call when I have a question about abc? Those are the kind of things that if you’re a new employee, just got hired to join a company, you would turn around and there’s someone there, your buddy, someone with experience that you can ask, but what would an agent do? So through self learning capabilities, what we’re making available in the Connect AI gateway is that ability to learn from prompts, right? So we’re seeing the prompt going to the model, we’re seeing the responses going back. We’re not saving any sensitive data on our side, but the learnings of what the user meant, and if the user is correcting the answer that they got from the model, that is a huge learning that we can keep. All those learnings are being curated as part of the context and the knowledge graph that is part of that so that everyone could benefit from them. So it’s enough for someone, somewhere, agentic or human, to make a mistake once, or learn from something once, so that that learning gets escalated into the context and there’s an opportunity for everyone to benefit from that. On top of that, we of course allow administrators to put in their own context so you can just go in and update certain things there yourself or to see a visualization of all the context that has been accrued for your enterprise as part of your enterprise context layer as well. So there are different, to summarize, different types of context that has to do with the data sources that you connect to, with the kind of questions that you ask, and with the learnings that you gain over time. And we’re able to benefit from all of those and construct an enterprise context layer that our customers can use with various data sources, data stores, clouds and model AI companies. I will just finish with one quick example of how this is extremely valuable to some of our customers. We had an incident a few weeks ago, an industry incident, I would call that Anthropic. Went down for a few hours and I was just on a call with one of our customers and they were freaking out. They said, look, we’re just connected to Anthropic at this point. Everyone is working with Anthropic. And right now for four hours, my engineers are not able to work because no one is willing to handcraft code anymore. What do we do? Right. And that reminded me the necessity of things like electricity or Internet connectivity. Right. It is your business responsibility and imperative to make sure that if Internet goes down, your business stays up. Something happens with electricity, business must stay up. Something happens with your model intelligence provider, what do you do then? Right. So through a gateway, we are able to help you make sure that your business stays up. So that is just one easy to understand example of the benefits that you gain by having that context and being able to point that into different directions.

Swapnil Bhartiya: Excellent, thank you. And since when you’re talking about that sensitive data, it also we need to talk about security and governance also, because it becomes critical once an agent starts to touch multiple enterprise systems and also start to take action on its own. So talk about security and governance here.

Raviv Levi: Yes, absolutely. And having spent more than a decade in the security space, I have a lot of appreciation to security aspects as well as governance and compliance and identity when it comes to agentic applications and even AI applications that are helping users to get the answers that they need, the answers that they seek. The architectural design, as mentioned previously, helps us make sure that we have a choke point. Everything goes through. We’re seeing every interaction with AI, we’re seeing every access to tools or data access, and we’re able to influence that. So this is the ideal place to set policies so that we can control who has access to what. A human that is trying to go and access the data source to the table row record level, an agent that is trying to do the same. The differentiation between an agent that is working on behalf of a human or an agent that is working with a service account because we shouldn’t have a dedicated identity and set of permissions for that kind of an agent. We can do all of that, we can track all of that and we can partner with security vendors. And this is part of what we’re announcing. We can partner with some security vendors so that they can see those events as well and they can act on them later on.

Swapnil Bhartiya: Now I want to just go a bit into the vids. How does Connect AI Gateway actually give organizations control over what an agent can access and do, right down to the individual record. Also considering that agents, they can break all boundaries, they can do whatever they want to do, as we have seen hugging face and such incidents in past as well. So how do you ensure that they do have that control?

Raviv Levi: We have a very sophisticated policy engine that we have deployed on the gateway, allowing our customers to predefine the policies and to enforce them. Now it’s important to say that agents don’t have access to that control plane, right? So the agents that are leveraging the Connect AI gateway are operating on a data plane. They’re going through through the gateway. So they’re going through the pipe, but they don’t have control over the pipe. That control is reserved to administrators on the customer side, on the CDATA side that are able to define who has access to what, which data sources, which tables, which rows, which records, under which circumstances. And as mentioned before, everything is also gets logged and exported as needed. So we have full visibility and observability.

Swapnil Bhartiya: Another very interesting or important topic when it comes to agents is accuracy, because that is what actually keep them out of a lot of critical cases and they just keep stuck in the kind of low stake use cases. So talk a bit about accuracy and where when it comes to AI agents, autonomous AI agents, no homo loop, we where does accuracy actually breaks down today and how much of that is a data or context problem rather than a model harness or governance problem?

Raviv Levi: Yes, that is a great question. And there are multiple levels that would contribute to an accurate answer that your agent could provide you at the end of the day. The first one has to do with the understanding of how your data is stored and where. And that is coming from metadata. And that is something that at CDATA our connectors and drivers have been providing for a while. So you have the opportunity to get the metadata out of the data sources that you connected to in order to understand get a picture, a high level picture of what is stored, where, and then using that metadata you can access the specific data, you can do your virtual join, you can do your multi system data queries and all of that. Being able to save that portion of the metadata as part of the context allows us to do that more accurately and faster. So that when someone is asking a question then we can go don’t need to go beyond the context to understand do we have an answer and where is this answer stored? And then we go there. And this is very important because LLMs specifically are extremely creative and they really don’t like to disappoint us. So if they don’t have clear instructions on what is available and where is it available, then they’re going to creatively come up with an answer that is absolutely wrong. And we’ve seen that time and time again. We have some benchmarks that we’ve been making public for people to try out. And I mentioned previously, we’ve seen results as significant as 65% accuracy with native MCP servers, all the way up to 98 and above percent accuracy when using the Connect AI gateway. So that is the first level where context is going to be able to help you with accuracy. The second level is with understanding the question. So up until now we talked about where is it stored and where do I go to get that piece of information. The new item that I’m bringing up now has to do more with what is the user asking, right? And that’s where you find all kinds of specific company lingo acronyms or confusing terms like enterprise revenue that could be translated into different meanings in different organizations. And there are other examples as well. But through the multiple layers of information that we’re making available for AI through our context, the AI is able to fine tune and understand exactly what the user is looking for and to provide that. And in the rare cases where the AI gets it wrong, then we’re able to learn from that. So it’s not just the semantic context that is able to drive accuracy. It grows with you, it learns with you, with the company and it’s always improving.

Swapnil Bhartiya: Another very, very critical discussion these days, and that is actually leading to the whole tokenomics as well, is cost. Because cost has kind of become a board level question with agents. Because every step you take, even if agent makes a mistake and you’re fixing the mistake, it’s going to cost you model choice, size of the model, how much quantity are the whole context window. I mean cost is touching almost every aspect. When you use AI agents and sometimes it feel like that’s not a sustainable model. So talk a bit about how much role, what role do data tool architecture context play when it comes to cost and what is CDATA doing to actually tame the cost, not to increase the cost through all this gateway that you’re building?

Raviv Levi: The first thing to keep in mind is that not all models are created equal and not all departments would need Fable or Astra level models in order for them to do their job correctly and efficiently and successfully. Using a gateway, such as the Connect AI Gateway allows you to set routing policies that are good, not just for model reliability, as we mentioned before, but also to decide who gets access to what. So that alone in and of itself is a huge impact when it comes to being able to optimize for cost. Another capability that you have is to understand what is the budget that I want to allocate for different departments for different individuals. What are the amount of tokens? Just get that basic visibility and observability on tokenomics, see who is doing what and then be able to get smart about it. Specifically when it comes to context, if an agent or an AI is trying to access a data source without prior knowledge, there are going to be a lot of tokens involved, right? It has to go in there, poke around, download a bunch of tables, a bunch of records, sift through them, see what’s interesting, and then if you ask a complex query that involves two or more data sources, then just multiply again and again and again the amount of tokens per the amount of data sources that you wanted to go out and get information from. But if you have a context layer, then that’s like a cheat sheet, a shortcut directly into, you know where the data is stored, you know what you’re looking for, you can just pull it out. So in our tests, what we’ve been able to see is a cost reduction of over 90% when it comes to the same queries trying to get the same information with the same starting conditions. One has the context, one doesn’t have the context, which is extremely significant and top of mind for practically every customer that I’ve been talking to.

Swapnil Bhartiya: Now if I ask you, you are making this announcement today, what does this architecture makes possible for enterprises that they could not have done easily yesterday? And how does it also help them give the confidence to move AI agents into more business critical workflows while also keeping accuracy, security and cost under control?

Raviv Levi: I think you hit all the right points and it’s really around confidence around being able to transform your organization to become AI native in a responsible way, both from a business risk standpoint, that’s governance and security and also from a cost standpoint. And that’s everything that has to do with tokenomics or token optimization and all of that. The only way for any organization to be able to do that successfully is by making sure first that they have an understanding of what’s going on. So you have to be able to see all the AI related traffic that goes through and then you have to be able to enforce certain governance and policies. And then finally, if you have the right AI gateway, then you’re also able to get the benefits of an accurate answer and an efficient access to data. Specifically for us here at CData, that has been our journey and mission for over 10 years, starting with the data connectivity, data replication. We hardly mention caching, but that’s a very important component as part of that. And we have evolved over the last few years to increasingly offer capabilities to our customers that would help them in their AI transformation. From connectors to AI agents and tools. Go through MCP server, MCP gateway and today we couldn’t be happier to launch our Connect AI Gateway with LLM Gateways, MCP Gateways, Agentic Gateways for A2A agent to agent connectivity kill switch and everything that comes with that. On top of all of the context and data connectivity technologies that we’ve been working on for a pretty significant time.

Swapnil Bhartiya: Right now for those organizations that are just getting started with agents. First of all, there are a lot of organizations, not a lot of, almost everybody is on the AI bandwagon these days, but because of, you know, everybody gets excited, they get on board and now actually a lot of organizations are rethinking, rearchitecting their, you know, whole AI architecture. So this is also right time for them as well as well as a lot of organizations who are starting their AI journey. What’s the practical first workflow where Connect AI Gateway will start showing value quickly versus over a period of time? Where should they get started? What are the touch points they should look at when they are looking at Connect AI Gateway?

Raviv Levi: Thank you. This is a great question. I understand that AI transformation could be overwhelming. There is a wealth of technologies and companies out there promising to fix the world and specifically your business. And it’s hard to understand where to get started. Working with quite a few customers and you know, I’ve been privileged, we’ve been privileged to partner with them on their journey. The easy first step I would recommend is first get visibility, understand what’s going on, who’s using what, how many tokens are being used by whom, and and get reliability. Start building the context so that you are not dependent on a single data store, a single model or a single cloud. Once you have those two things, then you can decide, okay, what’s next? What’s most important for me right now? Do I want to set policies? That’s easy because you have visibility and you can see exactly what’s going on. You want to save some money on tokens, and that’s easy. You can see what you can benefit by which measures that you may consider to deploy. You want to host a model locally and divert some of the traffic into this local model, you can do that as well. But it all starts from having visibility into what’s going on and starting to build that context layer within your gateway so that you’re able to leverage that for a variety of potential next steps after that.

Swapnil Bhartiya: You mentioned some use cases. Of course you’re announcing it today, but you work with your partners or beta testers. Is it possible for you to share some use cases without giving name? Or you can give name also where just to validate once they move to the gateway how their developers day to day life changed.

Raviv Levi: I can give. So without mentioning names, I should and maybe I should just give an example for myself. Part of, part of a good part of my last decade I, I spent with Cisco where I was an executive. I had a pretty significant team and, and business that I was responsible for. I had an executive assistant, I had a chief of staff. I had a variety of people just helping me go about my daily job. And I can tell you that in my current position, I don’t have less responsibilities, maybe even more than that. But all those functions of assistants and chief of staff that I used to be getting from humans, I’m getting from agents working side by side with agents. Whether it’s creating slides or reading tickets or opening tickets, reaching out to people, reaching out to customers, even preparing for this pleasant conversation, everything has been facilitated by an agent that has the right information at the right time because the agent is with me through all those meetings and has access to the same data sources that I have access to, has access to the emails and the messages and everything that I would have access to. And it’s only made available to me, made possible to me because we’re using the Connect AI gateway would not have been possible otherwise. And I can’t see myself going back at this point.

Swapnil Bhartiya: When we look at organizations who typically owns this gateway inside an enterprise IT platform team business units who are actually deploying agent. Where does the buck stops or it’s

Raviv Levi: like everybody, I think that most companies are starting with coding agents. That’s where we see the most value as an industry. We’re seeing the same here at CData, but increasingly we’re starting to see the next waves after that or agents are being used for more enterprise applications. So it’s not just about developers using agents to code, but everyone using agents in order to superpower their day to day job. Whether you’re in finance, marketing, sales, product management, it becomes the new standard of how you go about your work, how you access and transact with data and how you’re able to automate many of the things that you do.

Swapnil Bhartiya: Rav, thank you so much for walking.

Raviv Levi: Yeah, it’s been my pleasure. Thank you.

Swapnil Bhartiya: Yeah, it’s my pleasure as well. I just want to say that thank you for walking us through this and of course those who are watching it, please go and check. Connect AI Gateway at CData. And once again Ravid, thank you for your time and I look forward to chat with you again.