By An autonomous AI agent with no human in the loop gets a question about company data and has no map of what is stored where. LLMs are creative and do not like to disappoint, so without clear instructions they come up with an answer that is wrong. That is why accuracy keeps agents stuck in low-stakes use cases.
In this interview on TFiR, Raviv Levi, Chief Product & Technology Officer at CData, explains where agent accuracy breaks down and how metadata and semantic context fix it.
Guest: Raviv Levi, Chief Product & Technology Officer at CData
Show: The Agentic Enterprise
LinkedIn: Raviv Levi
Watch the discussion on YouTube
Here is what every platform engineer and AI practitioner needs to know.
Technical Deep Dive
Q: Why do autonomous AI agents give wrong answers about enterprise data, and how does metadata fix it?
Raviv Levi, Chief Product & Technology Officer at CData, says the first level of accuracy is understanding how data is stored and where, which comes from metadata. Levi explains that CData connectors and drivers pull metadata from connected data sources, and that saving it as part of the context lets the agent check whether an answer exists and where it lives, then go straight there for virtual joins and multi-system queries. In public benchmarks, accuracy rose from 65% with native MCP servers to 98% and above with the Connect AI Gateway.
“LLMs, in particular, are extremely creative, and they really don’t like to disappoint us. So, if they don’t have clear instructions about what is available and where it is available, they’re going to creatively come up with an answer that is absolutely wrong.” Raviv Levi, Chief Product & Technology Officer, CData
Q: How do AI agents understand company-specific terms and acronyms?
The second level of accuracy is understanding what the user is actually asking, Levi says. Company lingo, acronyms and confusing terms like “enterprise revenue” can mean different things in different organizations, and the layers of information in the context let the AI work out what the user is looking for. When the AI still gets it wrong, the system learns from the miss, so the context keeps improving.
“So, it’s not just the semantic context that drives accuracy. It grows with you, learns alongside you and your company, and is constantly improving.” Raviv Levi, Chief Product & Technology Officer, CData
Resources & Documentation
- Connect AI Gateway, CData‘s AI gateway product, referenced in the benchmark results discussed in this clip
***
👇 Click to Read Full Raw Transcript
Swapnil Bhartiya: Another very interesting or important topic when it comes to agents is accuracy, because that is what actually keeps them out of, you know, a lot of critical cases and they just keep stuck in the kind of low-stake use cases. So talk a bit about accuracy and where, when it comes to AI agents, autonomous AI agents, no human in the loop, where does accuracy actually break down today and how much of that is a data or context problem rather than a model harness or governance problem?
Raviv Levi: Yes, that is a great question. And there are multiple levels that would contribute to an accurate answer that your agent could provide you at the end of the day. The first one has to do with the understanding of how your data is stored. And where. And that is coming from metadata. And that is something that at CData, our connectors and drivers have been providing for a while. So you have the opportunity to get the metadata out of the data sources that you’re connected to in order to understand, get a picture, a high-level picture of what is stored where. And then using that metadata, you can access the specific data, you can do your virtual join, you can do your multi-system data queries and all of that. Being able to save that portion of the metadata as part of the context allows us to do that more accurately and faster so that when someone is asking a question, then we can go— don’t need to go beyond the context to understand, do we have an answer and where is this answer stored? And And then we go there. And this is very important because LLMs specifically are extremely creative and they really don’t like to disappoint us. So if they don’t have clear instructions on what is available and where is it available, then they’re going to creatively come up with an answer that is absolutely wrong. And we’ve seen that time and time again. We have some benchmarks that we’ve been making public for people to try out. And I mentioned previously, we’ve seen results as significant as 65% accuracy with native MCP servers all the way up to 98% and above accuracy when using the Connect AI Gateway. So that is the first level where context is going to be able to help you with accuracy. The second level is with understanding the question. So up until now, we talked about where is it stored and where do I go to get that piece of information. The new item that I’m bringing up now has to do more with what is the user asking, right? And that’s where you find all kinds of specific company lingo, acronyms or confusing terms like enterprise revenue that could be translated into different meanings in different organizations. And there are other examples as well. But through the multiple layers of information that we’re making available for AI through our context, the AI is able to fine-tune and understand exactly what the user is looking for. And to provide that. And in, in the rare cases where the AI gets it wrong, then we’re able to learn from that. So it’s not just the semantic context that is able to drive accuracy. It grows with you, it learns with you, with the company, and it’s always improving.
