Durable Execution for AI Agents: What Breaks Without It | Stephan Ewen, Restate | TFiR

Swapnil Bhartiya: When it comes to business applications, most backend logic doesn’t fail at the hard parts. It fails when the process dies halfway through a long chain of calls, leaving developers to handwrite retries, state machines, and cleanup jobs. Now AI agents are making it even worse because now software runs on its own for hours, making hundreds of models and tool calls that have to survive crash. Restate has a durable solution to this problem. They call it durable execution, which lets you write application logic while the runtime takes responsibility for recording progress, recovering from failures, retrying work, and recovering partial progress. And they just raised a $20 million Series A to make it a standard building block for backends and agents. And to talk about the funding and the company today, we have with us Stephan Ewen, co-founder and CEO of Restate. Stephan, it’s great to have you on the show.

Stephan Ewen: Thank you for having me.

Swapnil Bhartiya: It’s my pleasure, Stephan. Let’s first talk about the company. Talk about what problem area attracted you that led you to create the company. How old is the company and how is the company doing right now?

Stephan Ewen: The company— we started Restate 2023, early 2023, to tackle this problem of, back then that was pre-AI agents. We wanted to tackle the problem of unreliable backend code, really backend workflows. A lot of, we were all, most of the founding team was working on Apache Flink before. So generally in the space of You know, event-driven applications. We saw these patterns coming up again and again that folks who want to build like really scalable, reliable backends keep running into the same problems. The tools that existed make it really hard to write scalable, reliable backends. People keep solving the same problems all over again. It’s not just retries, but, you know, understanding how everything ties together between retries, between recording, intermediate progress between avoiding race conditions, split-brain situations, handling funky network situations, like how all of that comes together. It’s always been something where you spend a lot of the time. And we wanted to build a solution that makes this significantly easier. Now fast forward 3 and a half years, and aside from the usual backend problems and backend workflows, AI agents, like you already said, kind of entered the stage in our processes that suffer from those problems in a way more severe manner because they keep running for much longer. They have these non-deterministic elements in them with LLM calls. They usually interact with a lot of different pieces through their tool calls and integrations, and they basically need exactly the same tools. And here we are.

Swapnil Bhartiya: Now let’s talk about this funding around $20 million, which is led by, if I’m not wrong, Singular, Redpoint, and Capital One Ventures. Capital One Ventures. Talk a bit about ever since the company was created, what kind of traction you’ve seen and what is happening in the ecosystem, in the business that made this the right time to raise funding?

Stephan Ewen: We spent quite a long time getting the foundations of Restate in place. What I think makes Restate very special, quite different from almost any other project in the place, is how we’ve tackled this problem really from the Like from first principles, from the like bottommost layer. Restate has its own replication layer, you know, distributed log for fast durability and consistency, orchestration layer, scheduling, everything in a tightly integrated stack because that allows us to solve this problem very efficiently. We spent quite a bit of time getting these foundational pieces right. And then over the last year, we managed to to win a couple of pretty good customers across various segments. So Restate is used by, you know, both by financial institutions for workflows in that space and by companies like Replit to run their coding agent on it, by other startups that run, you know, backend workflows or AI workflows. And the right time, it felt the right time to raise because we had, you know, we had the foundations in place. We had proven out that this works. So this is something that That attracts customers across different segments. And the time is now to basically scale, go to market, scale engineering, and so on, and get it out there.

Swapnil Bhartiya: Now let’s talk about durable execution, which is a term we keep hearing, but depending on who you talk to, people may explain in a different way. What does it mean according to you? And when a developer adopts Restate, what changes in how their code actually runs before and after? So there are 3 questions bundled together. You can take them one by one.

Stephan Ewen: You’re actually right. I think durable execution is a term that is used by different people for different things. I think what it commonly means in a very abstract way is writing applications that have a sort of built-in way of persisting progress and restoring progress after a failure. And there’s lots of ways you can do that and, and, and very different, very different guarantees you get from that. There, there’s some solutions that just periodically take a checkpoint of your, you know, like these are the agents running and these were their inputs. Um, and they basically restart from that checkpoint and everything that happened in between is lost, but they call it durable execution. And then there’s solutions, um, More like, like Restate, that keep a very fine-grained journal of everything that happens. So at any point in time, when, when there’s a failure, the only thing you kind of, you lose is the very, very last in-progress operation because everything before has been journaled, can be replayed, it can be restored. So generally, durable execution means more built-in way to recover, recover your progress after failure. And then Then yeah, different sort of different degrees of like how fine-grained do you go there? This is very much also a question of like how much, like it’s very much a technology question. And in the end also how efficiently can you do this? Which kind of ties back a bit to what I said earlier. One of the things that I think make Restate quite unique is is the effort that we put in to make this an extremely lightweight thing. So the latency overhead of recording one of these, you know, journaling or recording one of these steps so you can restore them after failure is extremely cheap and low latency in Restate. And it allows people to apply it pretty broadly and in a pretty fine-grained way, which can help significantly simplify your application architecture and the way you write systems. If you can, you know, rely on much stronger, on a much stronger model that, you know, that can captures way more intermediate steps and allows you to not have to reason about, okay, there’s this whole window of things that might have lost. But you can really, you know, the smaller the window, the fewer operations affected, the easier it is to reason about the semantics of that.

Swapnil Bhartiya: Now let’s talk about how AI, as you also mentioned, a lot of things, you know, the way we have been doing it, it predates the AI. I mean, AI has been around for a while. You know, GenAI is this new thing here. AI agents, you know, they’re also kind of pushing everything to the next So let’s talk about from agent’s perspective. When an agent makes hundreds of model calls and tool calls on its own, what breaks without durable execution?

Stephan Ewen: So there’s a few things. Maybe that’s a good follow-up actually, even to your previous question. Durable execution systems like Restate don’t just really look at, okay, I’m persisting intermediate progress, but they also deal with, with a lot more. They also deal with sort of, let’s say, consistency in the presence of, let’s say, race conditions of other systems sending inputs, sending signals. What we see when we talk to our customers that are moving loads to Restate, what breaks is often is often one of, one of the following things. Either, either a long, a long-running agent that, let’s say, periodically persists the state, you know, loses a bunch of, of progress. Let’s say it calls to an LLM again. It gets maybe a slightly different answer than the last time because this is like, there’s a stochastic element to what the LLM is giving you as a response. And it might actually take a different path the second time. And in the end you get an inconsistent execution. Like, you know, one execution took one path, the second execution took a different path. The other thing is once you’re going into multi-agent systems, agents creating work and delegating them for others to subagents, these subagents keep calling back. Once you start scaling this and once you start having them run for longer and having them actually run across different failure scenarios, you get inconsistent views sometimes on the status of work on an agent reporting back. You get lots of race conditions that override and produce inconsistent context. That is one of the other things. I think what all of these share is that without a principled solution, it often gets very hard to understand what happened. You just see, okay, I have an output here. The output has like some inconsistent element here. I kind of simultaneously approved and disapproved some, something. And then you go and go back to the, through the, through the execution history and are trying to figure out what actually happened. Or you can actually see that, let’s say, a triage agent that, that delegated work to a couple of subagents got different results back from different attempts and basically now has conflicting responses. And yeah, all of these things have in common that once things go wrong without a strong foundation of like, for consistency, and part of consistency is recovering the right state, it gets very hard to like, first of all, give guarantees what’s happening. And second of all, like, you know, debug and fix the problems. when they’re happening. So to give you maybe the short answer is just like, you know, long-running, recovering long-running work, making sure you don’t end up in inconsistent code paths if you lose part of your work. And then, you know, in more complex situations, more multi-agent systems, the inconsistencies you get like just from, you know, distributed race conditions across different attempts, retries, executions, and so on.

Swapnil Bhartiya: Replit also moved its agent execution onto Restate with more than 10 times more durable action than before. Can you talk about what does an agent architecture look like when durability moves inside the agent loop instead of wrapping it in a workflow?

Stephan Ewen: Right. Okay, this is an interesting one. You can think about it the following way. When you write an agent, you can actually say, here’s, let’s say there’s a conversation step. One piece of input That, you know, you and your chat interface actually send to the agent and the agent basically loops until it has come to a point that it says, okay, you know, I’m done with whatever I was tasked or I need additional input. You can treat this as one, as one big unit. You know, once it fails, you basically redo the conversation step or you actually treat it as a workflow of substeps where you can say, okay, the inference call is a substep. The scheduling of a tool call is an individual step. Maybe even the guardrail before that, scheduling a tool call as a synchronous tool call versus a background call are different steps. Once you’re doing the second thing, you’re basically applying durable execution durability way more fine-grained. You’re basically recording not, okay, a conversation step has started with this input, but you’re recording, okay, this inference call happened, this tool call happened, another tool call happened. I received a signal that actually was steering me by providing additional context. I’m going back to another tool call. Here’s the next inference call that takes the previous signal into account and so on. You’re basically just decomposing it into a lot more fine-grained durable steps. The advantage that this has is it gives you, it creates a lot more work for the durable execution engine, but at the same time, it gives you a simpler, a much simpler consistency model for your application to reason about. Like, how do you, You know, what, what can you assume happened or didn’t happen? How do you, how you deal, how do you deal with crash scenarios? Um, the flip side is you need a durable execution engine that is built to keep the overhead of an individual step low enough and the efficiency of, of persisting one efficient enough so that this increase in, in work that you do recording, recording these more fine-grained steps. actually doesn’t become, you know, like prohibitive in the overhead that it adds.

Swapnil Bhartiya: Now let’s talk about latency and cost because that is becoming a very, you know, big topic these days. And I’m not talking about the whole tokenomics movement that is going on right now. Managed durable execution is often priced at tens of thousands of dollars per million durable actions, while queues and database costs around a dollar. How did you design Restate to close that cost and latency gap? Because that is really becoming a I think

Stephan Ewen: there was maybe a small typo. I think durable execution doesn’t cost tens of thousands of dollars per million durable actions, but tens of dollars versus the— Tens of dollars, okay. Tens versus databases on the order of—

Swapnil Bhartiya: Yeah, it could be flipped because when I look at all the token costs, I always look at the millions. Actually, people brag about, I burned that many million tokens. So yeah, I will correct that. But yeah, you are absolutely right about that. Slip of the tongue.

Stephan Ewen: Yeah, yeah, it’s scary.

Swapnil Bhartiya: But that’s not the one. Scary, yeah, please go ahead.

Stephan Ewen: So the way Restate makes this possible is by basically designing a system purpose-built for this task. I think pretty much every other durable execution engine sits on top of a database and basically uses the database to store progress, intermediate state, and events. Yeah. The thing about durable execution though is it’s fundamentally, I wanna say, kind of a simpler problem in the end. The, the thing you have to be able to do really fast is just append a journal entry, append, append a log entry that says, okay, here’s something that happened. Make sure this is durable. So, you know, when, you know, when we recover it, we can rely that this is there. This is the one operation you need to be really, really fast at. That is, that is inherently like a log operation. I wanna append something to a log. Yeah. So the way to make this efficient is not use a system that is built for way more complicated operations. More database systems are built for way more complex things than append an entry somewhere. That generality that database systems offer you comes at a cost. Like they have to do things in a certain manner in order to provide this way more powerful, way more general interface with SQL. If you actually strip the— if you strip it down to what you really need, I just need to be very fast at appending something, you arrive at, okay, what I really need is a fast distributed There’s a few, maybe, you know, a few things I glossed over. You kind of want these appends to be sort of conditional on, you know, on certain things like, you know, ownership, let’s say, of both the orchestration leader and maybe the execution didn’t change in the meantime. But, you know, you can still make it extremely efficient. So that’s the thing in the end that Restate did. We kind of optimized the path all the way from here’s the application doing a step and saying, I just need this step to be recorded all the way to Getting a replication quorum that says, okay, I’ve persisted this across a number of nodes and I can acknowledge that this will not be lost no matter what. Getting this path extremely fast. This basically pulls through the entire stack that we’ve built. The entire design is how can we spend as little work on this as possible? And that gives Restate these characteristics that when you do a durable step in your execution, you have a very low latency overhead, like within a few milliseconds, you’ll You’ll get the application to get an acknowledgement that this has been durably replicated, typically across multiple availability zones, and you can proceed. And that ultimately is, I think, is Restate’s advantage in both efficiency and latency. This deep, if you wish, integration across all layers from the protocol with which the application models this, speaks to Restate, to how the storage engine works, how the consensus engine works, how the orchestration engine works. to make that path extremely fast.

Swapnil Bhartiya: A lot of these, when we start working on these technologies, these processes, some become their own category. Do you think durable execution may become its own big infrastructure category or does it eventually kind of just disappear or kind of become part of how we build a system?

Stephan Ewen: I think there’s, There’s 2 answers to this. The first one is yes, durable execution, I think, is going to be a part of, I think, pretty much every application in the future because we’re seeing that more and more work in applications happens in this asynchronous long-running paths, especially if we assume that our AI agents are gonna play an ever bigger role in applications. But at the same time, I think the way many durable execution engines look today is not the shape in which it can become this piece. And there’s a few reasons for this. One is the efficiency we talked before. The other one is sort of the way, the shape in which many durable execution engines expose themselves, which is pretty much like workflow engines. You have to You have to kind of think in workflows, activities, workers, which is pretty different than I think many application backends are actually architected. That’s one of the other things that we, that is a very sort of core design decision of how we’re building Restate. We’re making this a system that where the paradigm is more you’re building durable functions. Durable functions that you assemble almost in the same way as, you know, people traditionally assemble distributed microservices, which gives you an abstraction that plugs into many backends very, very easily, requires less sort of like rewriting, rethinking of code, and gives you a lot more flexibility in the type of applications you can actually compose. So my take is, I think durable execution is gonna be key. But it needs to be in a different shape. It cannot be in the traditional workflow shape. It needs to move to a more flexible abstraction. Our bet on what that is going to be is durable functions, durable microservices. So that’s what we’re building.

Swapnil Bhartiya: Now, if you look at the way you have built custom storage and push-based dispatch model instead of task queues and polling workers, What should teams who are building infrastructure today learn from you breaking and bending some of these rules?

Stephan Ewen: You kind of almost answered this with breaking and bending rules. I think to some extent, you can always solve a problem, let’s say quickly, by assembling a few infrastructure pieces that you already have and then take it from there, but you’re inherently going You are going to inherit a lot of characteristics of those systems and you’re basically bound by what they can do in the case of every other system running on a database inheriting sort of the limitations and also the cost of using this way more general purpose engine for that. If you really want to take it to the next level and say, okay, we don’t want to think of this as something that is That is high overhead, that is very expensive. We wanna make sure that we can get the same guarantees, the same scalability, but an order of magnitude cheaper. You have to sort of like break with some of these, some of these established rules. Like, you know, when you’re building a system, you always start with a database and store your stuff there. And then you might have to go to the next step and say like, no, database really isn’t the right storage abstraction here. What is the right storage abstraction is a distributed log. So we need to, and it actually needs to be a very specific distributed log, like one that’s optimized for latency and a very specific pattern of writing to it. So let’s actually build this. This is how we make it really good. The same applies to the other part that you mentioned. Do you do, like, how do you actually connect these applications? You can get an easier start doing a worker task queue model. Mm-hmm. But you’ll first of all sort of like make this a requirement that every application now has to think of itself as like different activities running across different workers. And you’re going to inherit a certain complexity, also operational complexity and overhead from this. If you try to do it the other way around, if you try to say, okay, let’s make this— let’s think of this more as a broker that turns our existing code into durable functions, it’s a much harder thing to do. Like you have to think a lot about a lot more things. How do you think about failure detection in this case? How do you deal with flow control to not overwhelm downstream services? How do you deal with mapping capacity across different types of users and workflows that run across this? A lot of this becomes something you have to think really hard about. But when you actually figure it out, it becomes a very elegant and lightweight model that just handles a lot more centrally and allows the user to worry about fewer things, fitting more seamlessly into the deployment. So yeah, I think it is again about identifying those pieces that sound like, okay, common wisdom, but understanding where the limitations are, where do they come from, and understanding when it’s the right point to also break with them and say, okay, let’s Let’s rethink this. If we really want to take this to the next model, can we do this with this old approach or do we actually need a new approach here?

Swapnil Bhartiya: Thank you, Stephan. Before we wrap this up, I would also like to know quickly this, as we are talking about, this is a kind of crowded space. A lot of organizations are working in this problem. Why should someone look at Restate to solve this problem? What unique advantages do you have over your competitors?

Stephan Ewen: I think there’s a couple of ones that we see our customers mention when we ask them, okay, why did you pick Restate? The 3 biggest ones are, let me start with one we haven’t mentioned before. Restate is actually a very elegant, a very elegant compact system that if you wanna run it, if you wanna self-host it, if you wanna, want to run it in your infrastructure, possibly in a bring your own cloud solution, it’s very easy. It’s a perfect solution if you care about, let’s say, data privacy, about ownership of data infrastructure. If you feel okay, you know, We’re uncomfortable of running something that core against a third-party cloud service. Restate comes in this compact form, everything in a self-contained single binary, very flexible deployment models. That’s one of the big things. The second one and the third one are the ones we’ve talked about before. The form factor, you don’t write just workflows. You have a lot more that you can do. You can assemble basically durable functions and other primitives like virtual objects and so on into very flexible distributed applications and just actually build types of applications that you can’t really build easily on workflows. And the third one is the efficiency argument. If you have an application where you care about, okay, this is the overhead, overhead in latency, but also overhead in, you know, what does it cost actually, you know, resources-wise that I need to throw at the problem or money that I need to throw at the problem to actually run this, to actually sustain this load. Then Restate has a very attractive answer. So, you know, the flexibility and simplicity of how you run it, if you actually care about it running in your infrastructure, the flexibility of, you know, what type of applications can you build with it? How does it integrate into your stack? And finally, the efficiency, like how much does it actually add in overhead and cost you to run it?

Swapnil Bhartiya: Stephan, thank you so much for a great conversation. We, of course, covered the Series A, what durable execution actually means for developers, and why agents are pushing this from a workflow tool into a core runtime for backend. It was a great conversation. Thank you so much. And I would love to have you back on the show.

Stephan Ewen: Thank you so much as well.