By AI agents can query databases, retrieve documents, and call APIs. What they cannot do is absorb the unwritten rules, escalation paths, and organizational habits that experienced employees carry in their heads. That gap, the institutional knowledge gap, causes agents to produce generic answers, make avoidable errors, and require constant human correction. Without a mechanism to capture and share those learnings, every agent starts from zero.
In this interview on TFiR, Raviv Levi, Chief Product & Technology Officer at CData, walks through how the Connect AI Gateway constructs a living enterprise context layer that captures tacit and institutional knowledge, learns from agent and user corrections, and keeps agentic workflows running even when a model provider has an outage.
Guest: Raviv Levi, Chief Product & Technology Officer at CData
Show: TFiR
Here is what every platform engineer and AI architect building agentic systems needs to know.
Technical Deep Dive
Q: What types of context does an AI gateway need to give enterprise agents accurate answers?
Raviv Levi, Chief Product and Technology Officer at CData, identifies three distinct categories: metadata sourced from connected data systems, structured context imported from data stores such as Snowflake or Databricks, and tacit or institutional knowledge that exists only in the minds of experienced employees. The first two are addressable through standard connectors. The third requires a fundamentally different mechanism because it is never written down and cannot be imported.
“There are different types of context that has to do with the data sources that you connect to, with the kind of questions that you ask, and with the learnings that you gain over time.”
Raviv Levi, Chief Product & Technology Officer, CData
Q: Why is institutional knowledge a specific problem for AI agents that it is not for human employees?
When a new human employee needs to understand an unwritten process, such as how to file an engineering ticket correctly or who to contact about a specific issue, they can ask a colleague. An agent has no such colleague. Levi frames this as a structural gap: the agent has no way to learn conventions that exist only through person-to-person transmission, so it will produce incorrect or incomplete outputs until that knowledge is surfaced somewhere the agent can reach.
“There’s no one there by the agent to teach the agent about those things that are not written anywhere.”
Raviv Levi, Chief Product & Technology Officer, CData
Q: How does the Connect AI Gateway capture and store institutional knowledge without exposing sensitive data?
The gateway observes the prompt sent to the model and the response returned, then extracts the learning when a user corrects or refines that response. CData does not store the sensitive content of the prompt or reply, only the derived learning about what the user intended and what a better answer looked like. Those distilled learnings are curated into the enterprise context layer and made available to every subsequent agent or user interacting with the same system.
“We’re not saving any sensitive data on our side, but the learnings of what the user meant, and if the user is correcting the answer that they got from the model, that is a huge learning that we can keep.”
Raviv Levi, Chief Product & Technology Officer, CData
Q: How does shared learning in a knowledge graph benefit all agents and users, not just the one who made a correction?
Once a learning is captured, it is escalated into the shared knowledge graph that underlies the enterprise context layer. Every agent and every user that subsequently interacts with the gateway benefits from that learning without needing to repeat the same mistake or correction. Levi describes this as a compounding advantage: a single correction by one person or one agent raises the baseline quality for the entire organization.
“It is enough for someone somewhere, agentic or human, to make a mistake once, or learn from something once, so that that learning gets escalated into the context and there is an opportunity for everyone to benefit from that.”
Raviv Levi, Chief Product & Technology Officer, CData
Q: Can administrators inspect and manually update the enterprise context layer?
Yes. The gateway exposes a visualization of all accrued context so administrators can audit what the system has learned. They can also add or edit context entries directly, without waiting for the self-learning mechanism to surface a correction. This gives teams explicit control over the knowledge graph alongside the automated learning process.
“We of course allow administrators to put in their own context so you can just go in and update certain things there yourself or to see a visualization of all the context that has been accrued for your enterprise.”
Raviv Levi, Chief Product & Technology Officer, CData
Q: What happens to agentic workflows when a model provider like Anthropic goes down?
Levi describes a direct customer incident in which Anthropic experienced a multi-hour outage. Engineers at a CData customer who had connected exclusively to Anthropic were unable to work for the duration because they had built no fallback. Levi compares this dependency to electricity or internet connectivity: organizations treat those as infrastructure that must be resilient, and AI model access now belongs in that same category.
“Something happens with your model intelligence provider, what do you do then? Through a gateway, we are able to help you make sure that your business stays up.”
Raviv Levi, Chief Product & Technology Officer, CData
Q: How does routing enterprise context through a gateway provide resilience against model provider outages?
Because the enterprise context layer lives inside the gateway rather than inside any single model provider, the gateway can redirect traffic to an alternative model when the primary provider is unavailable. The accumulated context, including institutional knowledge and prompt learnings, travels with the redirect rather than being lost. This makes the gateway the continuity layer between enterprise knowledge and model intelligence, independent of which model is currently serving requests.
“We are able to help you make sure that your business stays up. So that is just one easy to understand example of the benefits that you gain by having that context and being able to point that into different directions.”
Raviv Levi, Chief Product & Technology Officer, CData
Resources & Documentation
- CData Connect AI Gateway, enterprise AI gateway providing context management, institutional knowledge capture, and model provider resilience
***
👇 Click to Read Full Raw Transcript
Swapnil Bhartiya: Now, as you rightly explained that the whole Connect AI Gateway is built around the idea of bringing enterprise context to agent. Can you talk about context may mean different thing in different context altogether. What does context mean here and how does gateway build and keep improving that context over time? Because company data continues to evolve as well.
Raviv Levi: Yes, that is a great question. We talked about multiple types of context so far. There’s the metadata that is coming from the data sources. There are some types of contexts that could be imported into the gateway, whether coming from your data store, like Snowflake Databricks or others. But there’s another type of context that is really, really important. There’s a lot of conversation in the industry these days around a concept that’s called tacit knowledge or institutional knowledge. This is the kind of thing that employees, new employees joining your company would have to learn. That’s something that’s not really written anywhere. Someone would need to teach them. That creates a problem for agents, as you can imagine, because there’s no one there by the agent to teach the agent about those things that are not written anywhere. And only employees would know one. One example could be how do I open a ticket? How do I write down a ticket for the engineering team, There are certain nuances or certain things that would go there, or who do I call when I have a question about abc? Those are the kind of things that if you’re a new employee, just got hired to join a company, you would turn around and there’s someone there, your body, someone with experience that you can ask, but what would an agent do? So through self learning capabilities, what we’re making available in the Connect AI Gateway is that ability to learn from prompts, right? So we’re seeing the prompt going to the model, we’re seeing the responses going back. We’re not saving any sensitive data on our side, but the learnings of what the user meant, and if the user is correcting the answer that they got from the model, that is a huge learning that we can keep. All those learnings are being curated as part of the context and the knowledge graph that is part of that, so that everyone could benefit from them. So it’s enough for someone somewhere, agentic or human, to make a mistake once, or learn from something once, so that that learning gets escalated into the context and there’s an opportunity for everyone to benefit from that. On top of that, we of course allow administrators to put in their own context so you can just go in and update certain things there yourself or to see a visualization of all the context that has been accrued for your enterprise as part of your enterprise context layer as well as. So there are different, to summarize, different types of context that has to do with the data sources that you connect to, with the kind of questions that you ask, and with the learnings that you gain over time. And we’re able to benefit from all of those and construct an enterprise context layer that our customers can use with various data sources, data stores, clouds, and model AI companies. I will just finish with one quick example of how this is extremely valuable to some of our customers. We had an incident a few weeks ago, an industry incident, I would call that Anthropic. Went down for a few hours and I was just on a call with one of our customers and they were freaking out. They said, look, we’re just connected to Anthropic at this point, everyone is working with Anthropic. And right now for four hours, my engineers are not able to work because no one is willing to handcraft code anymore. What do we do? Right? And that reminded me the necessity of things like electricity or Internet connectivity. Right? It is your business responsibility and imperative to make sure that if Internet goes down, your business stays up, something happens with electricity, business must stay up, something happens with your model intelligence provider, what do you do then? Right? So through a gateway, we are able to help you make sure that your business stays up. So that is just one easy to understand example of the benefits that you gain by having that context and being able to point that into different directions.
