By Enterprises are discovering that deploying AI agents in a controlled demo environment and operating thousands of them autonomously in production are two entirely different engineering problems. Integration complexity multiplies with every agent added, reliability ceilings emerge well before acceptable thresholds, and the security stacks that protect enterprise data were never designed for agentic workloads. The gap between an 80% success rate and the 99% reliability required for true autonomous operation is not a minor tuning problem. It is a fundamental architectural challenge.
In this interview on TFiR, Mario Moscatiello, VP Growth at Airbyte, breaks down the four core technical blockers that prevent enterprises from running AI agents at scale and explains what needs to be solved before autonomous systems can be trusted in production.
Guest: Mario Moscatiello, VP Growth at Airbyte
Show: TFiR
Here is what every platform engineer and AI infrastructure team needs to know.
Technical Deep Dive
Q: What is the biggest technical blocker stopping enterprises from running AI agents at scale?
Mario Moscatiello, VP Growth at Airbyte, identifies integration complexity as the primary blocker. Every data source an agent reads from or writes to requires solving for OAuth, pagination, rate limits, and error handling independently. When thousands of agents are simultaneously reading from and writing to hundreds of systems, any upstream API change, such as Salesforce updating its API, can break agent workflows at scale. The effect is multiplicative: more agents connected to more systems means exponentially more integration failure points to manage and maintain.
“If you have a thousand agents that are reading from hundreds of systems, it’s a multiplier effect. And that’s the number one cause that we see today.” — Mario Moscatiello, VP of Growth, Airbyte
Q: Why is an 80% agent reliability rate a problem for enterprise production deployments?
Moscatiello points out that most enterprises currently find their agents functioning correctly only 70 to 80% of the time, and explains why this is a hard ceiling that makes autonomous operation unviable at scale. Unlike deterministic code, where systems exist to catch even minor response drift, agent errors can be so small they go undetected. Moving from 80% to 99% reliability is not incremental tuning. It requires dedicated evaluation frameworks, regression testing, and performance monitoring built specifically for non-deterministic outputs.
“Working 80% of the time doesn’t really work at scale, especially because agents sometimes make errors that are so small they are not even detected.” — Mario Moscatiello, VP of Growth, Airbyte
Q: What is context degradation in AI agents and why does it cause production failures?
Context degradation occurs when agents are provided with too much data simultaneously and cannot distinguish what is relevant from what is noise. Moscatiello describes a common failure pattern where teams assume that giving agents access to all available data will improve output quality, but the opposite occurs: agents produce incorrect responses because they are unable to prioritize effectively within an oversaturated context window. Solving this requires deliberate context scoping and retrieval strategies rather than broad data access.
“The agents can produce wrong responses because they have too much context and they don’t know what’s important and what’s not important.” — Mario Moscatiello, VP of Growth, Airbyte
Q: Why is the current enterprise security stack insufficient for agentic AI workloads?
Moscatiello identifies governance and security as the fourth major blocker, noting that existing enterprise access controls and security frameworks were designed for deterministic code executed by a small number of human-supervised processes. Thousands of autonomous agents operating simultaneously across sensitive systems expose gaps that current security tooling cannot address. He describes the security stack for agentic workloads as immature, meaning enterprises deploying agents at scale are doing so without adequate controls for access management, audit trails, or policy enforcement built for non-deterministic, high-volume agent behavior.
“Security and governance is not built for thousands of agents running. The security stack is immature.” — Mario Moscatiello, VP of Growth, Airbyte
Q: What role do evals play in improving AI agent reliability for production systems?
Moscatiello points to the growing industry focus on evals as the primary mechanism for tracking and improving agent performance over time. Evaluation frameworks allow teams to monitor regression, measure output quality against expected results, and systematically close the gap between current reliability rates and production-grade thresholds. Without evals, teams lack visibility into whether agents are degrading, improving, or producing silent errors that fall outside existing detection systems built for deterministic code.
“That’s kind of why you hear a lot about evals, making sure that we can look at regression and the performance and so on and so forth.” — Mario Moscatiello, VP of Growth, Airbyte
Resources & Documentation
- Airbyte, open-source data integration platform for moving data from any source into your data stack, including AI and agent workflows
- Airbyte Documentation, official documentation covering connectors, deployment, and API integration
- Airbyte on GitHub, open-source repository for connectors, contributions, and self-hosted deployment
***
👇 Click to Read Full Raw Transcript
Swapnil Bhartiya: As we all know that a lot of companies are experimenting with AI agents, but going from a slick demo that they saw in the boardroom or with developers to production system is still hard. Can you talk about what are some of the biggest technical hurdles that keep enterprises from running thousands of autonomous agent at scale?
Mario Moscatiello: Yeah, I would say, like, of course, in the case of. I would say in the case of Airbyte, what we see is that the integration, plumbing, tax is real. Every data source that you’re trying to get into the model means that you have to solve for OAuth pagination rate limits, error handling, and all of that stuff. And so it’s stuff like the more agents you have running, the more systems they’re reading and writing to. So if you now have agents writing back into Salesforce the moment Salesforce updates their API, like, it might break something. And so, like, you have to. Integrations are still, you know, like the number one issue in that sense, based on what we see, because it’s very hard. And if you have a thousand agents that are reading from hundreds of systems, it’s a multiplier effect. And so, like, that’s the number one cause that we see today. I would say the second thing is that at most companies are able to have agents work 70, 80% of the time, which is kind of like a bad ceiling. Because when you’re running, if your company has been like running autonomously, or if you want your company to run autonomously, you know, working 80% of the time doesn’t really work at scale, especially because I think that agents sometimes make errors that are so small that are not even detected. And so, you know, engineering has built a lot of guardrails when it comes to errors and when it comes to responses that are not what they expected. Especially with deterministic code, right? You run code and you expect a response if there is a drift. There is systems that are put in place to detect even the smaller errors. When it comes to agents, it’s a completely different pair of ants. And so I think that 20, 30% of like, hey, can we get the agent to work 80% of the time to working or 90% of the time to working like 99% of the time. I think that’s going to be a hard problem to solve. And that’s kind of like why a lot of you hear a lot about evals, making sure that we can look at regression and the performance and so on and so forth. So I think that’s the second big problem. I think the third one is context degradation, which is when you just flood the agents with a lot of context, and you think, like, oh, the agents should be able to see all of our data, and so on and so forth. The agents can produce wrong responses because they have too much context and they don’t know what’s important and what’s not important. And so I think, to sum it up, those are the three main causes. But the fourth one, to your point earlier, is governance. And we go back to security Access has been built for deterministic code and for a few people within an organization managing highly secure information. But Security Guard lets governance is not built for thousands of agents running. And so I think the security stack is immature. And so that’s going to be also like a big, in my view, a big point to solve.
