Swapnil Bhartiya: You would think that mobile apps would be as easy to debug as backend services, but they’re not. When something breaks on a phone, teams often wait for the next release just to find out what went wrong. Now add AI engines into the mix and most obsolete tools were not built for them at all. But that is not the case. Bitrift says that they have cracked both problems, launching the world’s AgentIQ mobile observability platform. And today we have with us Peter Morelli, CEO and co founder of BitDrift, to talk about that. First of all, Peter, it’s great to have you on the show.

Peter Morelli: Thank you for having me.

Swapnil Bhartiya: It’s my pleasure to host you here. Let’s just talk about the company. So talk a bit about how the company and what led to the creation of the company, because you’re a co founder. So let’s talk about why you created it.

Peter Morelli: I’m happy to do so, yeah. So my previous job was the VP of engineering at Lyft, and Lyft, obviously 99% of its business comes over the phones. And so we ran into a lot of issues trying to understand a great customer experience on their phones. And so probably we worked on this problem for almost six years trying to understand and fix the problems that you can get. And then about three years ago we spun this out as a company. So this is the technology we’ve been working on for almost 10 years, trying to understand and deal with the scale of both mobile and its very unique architecture. I can go into a little bit of that, but I think that’s sort of the origin of the company.

Swapnil Bhartiya: Before this interview, we were talking about the problem areas and it’s like when it comes to mobile observability, it has been more or less treated as an afterthought for years because most tools were built for servers and backends, they were not built for mobile, they’re kind of retrofitted for mobile. So talk a bit about what makes mobile observability fundamentally different, challenging and why does the same tool that works on the server or backend doesn’t work here. Why that approach fails, that’s a great question.

Peter Morelli: So generally, observability tools have been built this way, where you have an agent or a collector or an SDK on the actual server, you collect the data through some sync and you push it to a central aggregation point. That’s sort of how all most observable. That’s where you do the graphing and the aggregation and sort of the analysis that observability systems are built for, and they’re fantastic. If you’re in a data center, the problem happens when you try to put that on a device. And there are a couple of things that happen is one, they’re all built with those assumptions that you’re in a data center. You assume that you have a lot of bandwidth. You assume that that bandwidth is reliable. You assume you have a lot of CPU and storage and that it’s always running. None of those are actually true for mobile. In fact, the normal case for mobile is that you don’t have a lot of bandwidth. It’s unreliable, especially as you move around, certainly if you’re in a car or if you go in a tunnel, those types of things. And so what happens is that a lot of times when our customers had the worst experiences, we were missing that most of the things they would try to collect some data, push it up there, they didn’t have a network connection, they drop it on the floor. And so we were only seeing the good conditions and the good experiences our customers were having. We were fixing the wrong problems. The other problem was happened was when we scaled most server solutions, like a couple hundred thousand servers, maybe there are more extreme cases, but generally you’re in the 50 to a couple hundred thousand range. We had tens of millions of mobile devices every day sending data. I think Lyft at its peak was pulling in 45 to 50 terabytes a day just from the phones. And so when you’re dealing with that sort of scale, that’s an immense amount of cardinality. It’s an immense amount of data that’s coming in if you’re trying to do a lot of logging. And so what we found was that it either was a scale problem or it was a cost problem, since everything is cost based on ingestion. And so most people solve this even to this day by sampling pretty heavily, like usually 1 or 2%. So you’re getting 1% of only the good conditions. And that’s really a very, very small slice of the problems your customers are having. So that’s sort of why we built something a little bit different.

Swapnil Bhartiya: And as you’re talking about your tenure at Lyft as well, and that’s where, based on their experience that you mentioned, that’s what led to the company. Can you talk about what were some breaking points, the movements where existing observability tools simply could not give you what you actually needed, not from your end to solve the problem, but from also the customer’s end that what kind of experience they get when they hit that debug button and then nothing happens about. Not only that, but also the scale at which Lyft or modern companies work today.

Peter Morelli: Yeah, most modern companies, I think the sampling that people do starts at even as 200,000 devices, it’s actually a very small deployment when you’re looking at modern companies and the customers that we actually work with a lot. So the breaking points tend to be there’s another sort of thing, like in observability, a lot of times your failures tend to be clustered. A lot of times, especially in a data center, that switch that goes out, that one port is burned. The canaries have the one build, but no one else does. So your errors tend to be clustered in areas. Whereas in mobile world, I call it like actually dealing with reality, the physical world, you have all these different variables. And so you don’t have this very nice like, you know, fat like tail of the curve, you have a very fat set of errors that are happening. So you don’t have like all your errors cluster, they’re spread out of there. And then what I mean by that is like let’s say you’re using a foam inside, like you’re ordering something, you’re getting a car, whatever it is. You walk outside, you’ve switched from wi fi to 5G, even those both great connections, you’ve swapped network connectivity, you may drop something, you might walk behind a tree and that happens. You were driving down the street at 50 miles an hour and your one cell tower that was working is now saturated, even though you have five bars. And that’s before you add in the different hardware devices that you’re all deployed on. The different OS versions, the different app versions, the different feature flags you. So the world in which your customers are having experience is incredibly varied and all the different combinations are very difficult to debug. And so I view it as like a world of edge cases, right. If you’re trying to manage the tail and the P99 experiences, that P98, P99 is very, very long and very, very fat. And so if you’re sampling and only getting the good conditions, you’re missing most of your customers problems. And that was the breaking point for us is that happened too many times over and over again. We would try to figure out a customer’s problem, couldn’t replicate. It took us forever because the mobile release cycle was 10 to. And if you’re waiting 10 days for each set of logs to come back, each new set of things, it’s a very Long debugging cycle. So that was one of the other problems that really broke. And after years of asking the vendors to try to solve this with us, and I’m still not doing it to this day, I think that’s what was sort of the breaking point for us and where we said we need to invest a lot of engineering time in this.

Swapnil Bhartiya: I would talk about company also a bit, but I also want to talk about the AI side. You folks launch BitRift AI, which gives AI agents kind of programmatic access to real time telemetry from actual user devices. Can you walk us through what that means in practice? What can an agent do today that a human engineer could not have done before?

Peter Morelli: Yeah, to do that. Let me describe a little bit about how Bedrift works, because I think it’s pretty integral to the system. So I mentioned before that most observability systems, they collect data, aggregate it into a central point. We do it very different. We process at the edge almost exclusively. And so we create a series of ring buffers on the phone or we collect as much telemetry as we can get stuff that we collect. But also whatever as a developer you put into that that sort of collects locally, we don’t persist it by default. The other part of the system though, connects to every device in real time and says, hey, here are the conditions I’m interested in. From the server side, you’re able to say, I’m interested in this cohort or this conditions or this customer’s had this error four times or more. But only in San Francisco you can have a lot of flexibility of saying, here are the conditions I care about. And you can either pull the data server side to actually do it from matching phones and analyze it, or you can process it at the edge. Say, I just want to count how often that has happened. You can metricize all the logs that are happening. So now you have a system that is sort of freed from the mobile release cycle. It’s also freed from the ability of should I log or not? Right. We’re not charging you based on ingestion because we’re storing and processing it locally. And so all of a sudden you have this sort of distributed network and a live view into it and saying, I now want to pull data when I have conditions and problems or things that I want to investigate. So that’s great for a human when they’re debugging a problem. It’s even better for agents. Right? Agents work in very iterative loops. Right. You sort of say, hey, why is my onboarding Funnel broken, you can go and say, hey, well I’m going to start investigating. The agent does the same very similar thing. If you were stuck with other observability providers, you would have to wait 10 or 15 days between each loop as they ask and get for more information if you don’t have that up front. And so for the first time we’re actually able to, to give people access to this data that they never had before. Remember this was sampled at 1% and only the good conditions. Now you have unsampled views of all this data and you can give that to your agents and they’re able to loop iterately, ask for more information, deploy more observability, get more details, do more analysis, all without having to wait for that mobile release cycle. And so agents are actually quite good at doing this. Humans generally like, you know, focused on the most, you know, the highest priority problems. We have a customer that says like we typically only address the top 20 or 30% of problems. Once we, you know, added bit drift AI and start started using the agents, we were able to get to almost 80 to 90% of the problems. Remember I said it’s that very long tail of problems. Right. And so being able to address all of that proactively is something that I think is a really unique, you know, sort of thing that is a combination of both our product. But really what you know, AI has been enabled a lot of, a lot of engineers to do. So I love that productivity again, I love the actual, we’re making the customer experience better for a lot of customers.

Swapnil Bhartiya: That’s great, Excellent, thank you. And when it comes to observability, building it for humans means of course, dashboards, charts, the whole drill down workflows. But building it for agents is a totally different design problem, totally different governance problem as well. So talk a bit about this fundamental difference, the challenges that you folks faced at the same time, how your potential customers can use them because they are so used to. But now today everybody’s getting used to agent based agent workflow. So that’s not a challenge. But I do want to understand this difference technically, philosophically, culturally.

Peter Morelli: Yeah. So I think we designed the architecture in a way that I think was built for built in layers. My co founders and I have built a lot of platforms at many different companies and so we approach this the same way, which is to build it in terms of layers. And so you know, our UX is driven off our API and so we have at the bottom layer our sort of API. It’s now a Public API. On top of that, we have different interfaces. One is the user experience that everyone sees. We’ve also created a CLI that makes using that API easier to use. Both of those are great for humans. All three of those are great for humans, depending on where you want to hook in. On top of that, we’ve added agent skills, and so now we can teach AI agents how to actually use these tools that are provided. So you now have a system that has nice lay. You can actually interact with it at every sort of whatever choice of layer that you want to. And so the way we built it, I think was built with programmatic access in mind, which made it very nice to actually port over and have AI agents start to take advantage of it. So from our perspective, porting over to it, I think took a little bit of thought. We redesigned some APIs. I think we redesigned the output so it was much more efficient. The CLI was built really with token efficiency in mind so that the common things would not take 1,000 API calls to do. So we think of the agents as an actual user of our system. It’s a separate Persona, it’s a separate user. And as we design them, but the whole architecture is sort of stacked together in a way that allows both humans and agents to really use it.

Swapnil Bhartiya: Well, can you talk about this whole ring buffer concept? What it is and what does it also mean for capturing everything locally without killing performance and battery? How do you supposed to do that?

Peter Morelli: Yeah, yeah. So the core system is like to use our system, you take an SDK that we provide you and you install it into your application. That application then sort of can collect telemetry and use that SDK. The SDK, what it does on initialization, it creates a ring buffer. It’s sort of a fixed size. We don’t want to grow with unbounded things. Devices has limited sort of capacity. And so we give a limited amount of memory. And then as you write and collect telemetry, as it gets to the end, it sort of starts at the beginning. That’s the ring part of it. And so, you know, you have a fixed size of memory that you’re allocating to collecting telemetry that actually can grow and shrink. We have customers who, you know, are used on kiosks or point of sale. They make that 200 megabytes, right? Or 800 megabytes. We have also customers on mobile devices that do it at five. And we did a lot of engineering, like a lot of work to make it performance on mobile devices, even low end devices. So you have 16 milliseconds before a human will notice a freeze or a lag or something that is going on in the ux. So everything is done in the background. Everything is done as async as we can. The core of it is written in rust, so we get a lot of performance and memory benefits from that. So a lot of the work is really focused on making sure we can collect a lot of data without affecting the user experience. The ring buffers are nice. We memory map them to disks so that they persist between crashes or, you know, someone force quitting something or the OS reaping you. So you have this data beyond crashes because it’s one of the nice features of our thing. So let’s say you don’t have a network, maybe you don’t use the app for a week. You still have the access to the data and we collect it and hydrate it afterwards. So the ring buffer allows us to say to engineers, you can collect as much data as you want, you won’t blow out the device, you won’t cause performance issues in your app, but you can collect a lot of data. And that five megabyte buffer, or even for a chatty app, is usually good for days of data. I mean, obviously if you’re putting like 20 megabytes a second of data in there, it will run out. But generally we’re giving people a buffer that allows you to play with data and add it. So most people now you don’t have to choose, should I log this or not? You just log everything. And we sort of manage that. Sort of. The persistence we manage making sure you don’t blow out the device with that ring buffer.

Swapnil Bhartiya: Since, as you mentioned that you gave beta access to customers, can you talk about what kind of improvements they saw when they were testing it and what that feedback is led to? What you were expecting one change about like during that beta phase, because we are launching it today, so now we’ll start seeing in production, but what kind of feedback you receive from them, where they’re like, hey, you know, this is what changed today after going to this beta and start using it versus what I was doing yesterday.

Peter Morelli: Yeah, that’s a good question. So our customers almost universally loved it. I think it really improved the efficiency in a couple of different ways. One was in the integration and in the addition of telemetry. Right. Instead of saying, hey, let’s manually add log lines or manually adding things, you could tell the agents, hey, I want to use bitrift, and then add the logging that we Want to. So stuff that used to take a couple of weeks or a month was compressed down to days. The other one was the system we have of saying, hey, go out and find matching devices and matching data conditions or cohorts and pull it back. That is its own sort of like set of information and it’s a cognitive load that people and developers had to learn. And it’s pretty intuitive from a UX perspective. But being able to say and ask higher level questions like why is my onboarding funnel broken? Right. Where are my performance issues today? Those high level questions were able to be mapped down and like broken out into investigations without humans having to break them down and then deploy them and learn them. And so stuff that used to take, even during POCs, we would see people like write like 10 of these, you know, really high level questions and be able to sort of, you know, you know, spin them up and get them running within a day. So I think from a speed of execution, a speed of iteration, lightning, like huge changes. And so what that allowed one was fixing the problems a lot faster. So MTDD and mtdr like really dropped, but it also allowed them to address a lot more of the problems. I mentioned that example of someone who, you know, probably best case was hitting 20 or 30% of the customer issues that were popping up. The highest priority ones they were able to address up to 80 and 90%. And that’s a huge shift in both the customer support that they have to deal with, but also in the experience the customers are having.

Swapnil Bhartiya: Perfect. Awesome. Thank you. Now, if agents can query reality on mobile devices today, where does this go? I mean, it’s very hard to actually predict where things will head. But if you look at also these days, I feel that more and more people are using mobile devices. I feel that even the AI workloads, I don’t know, it’s very hard to predict actually these days. But if I say because of bitrift, what you folks are doing, how it will redefine the whole debugging the whole mobile platform and things are becoming more and more. So I’m not asking how things will look like in 10 years from now or two years from now, but just how do you see the, the evolution of this space?

Peter Morelli: We’ve already seen in the last year just a huge like increase in the volume of code that is being generated. Right. I think the agents have especially made generating code and generating applications really, really much easier and cheaper than before. I think the number of app submissions has more than doubled. I think in the last year we have app generators that are doing this. I see more and more applications being built both internally and for external applications. And so I think the volume of this sort of deployment will only increase. You’re only going to see more and more compute at the edge, right? Like now your TV is actually running Android under the covers, right? You see your kiosks at any airport that you go to are running Linux or Android or some variant of those. So you’re only going to see more and more cases where there are devices that have compute, relatively powerful compute compared to early days of like sort of Iot. I think the problem that we’ve talked about, being able to access all of the data, even in adverse conditions, even with timing challenges, is only going to get more and more. And so I think this for the first time allows you access to that stuff that before that data was really only done server side. You’re really only trying to interpolate it and trying to figure out what was happening from a very small sample. So I think that is the big difference. The other one is the sort of real time nature of it, right? Like if you think mobile phones take a long time to Update, like that 10 or 15 day cycle to get a new app version out, how often do you update your tv? Like every six months? Never. A lot of people don’t do it because they don’t want anything to break. So the problem that you face where your observability has to be more runtime versus more compile time, that’s the big difference and that’s the big shift that this architecture allows. You’re not having to wait for pushing code out to be able to understand problems and debug them and try to fix the, and maybe mitigate them. And I think there’s only more and more code. The process of supporting and maintaining software where a lot of people are focusing is not nearly as developed. I think Peewee will continue to develop on that, but you’ll need architectures like this to actually feed them. And that’s where we see ourselves as fitting.

Swapnil Bhartiya: Of course, we are going through all this geopolitical crisis. A lot of regulations are happening in Europe, in the us a lot of things are happening. Chinese, AI Usai, what kind of impact you see will be there? Because the mobile market is a global market, it’s not a regional market. IPhone user, globally, Samsung, you’re globally. So first of all, how do you see the impact of this changing geopolitical crisis on this space? And you feel that, you know, actually this is the right thing. The Right time by giving users, developers more control, more power to solve the problems. What do you think?

Peter Morelli: Yeah, it’s a good question. I think it’s complex. I think for Bitra specifically, we built the system to be very friendly to data and privacy sort of considerations. By default it collects no pii. We don’t prevent customers from doing. There’s no new permissions that you have to ask us. We generally defer to the application developer about what data they want to collect. Even when they want to do matching on user IDs, we hash them so we don’t have access to that data. So, you know, our session replay is a wireframe. It’s not actual, you know, screenshots. We’ve done a lot of things to make sure that, you know, we are, we are a good, good citizen and in fact a great citizen in terms of like privacy and data on the server side. We’re able to, you know, data home, wherever customers need us to be. And so we can put, you know, the customer data they collect in their environment. We can run our whole stack in their environment. We can put it in the EU or other places. I will say we have not done any business in China that would probably be more problematic. But generally we built a system that’s very flexible in terms of what data is collected and where it can be put. The nice thing about our system is that most data stays on the device, the customer’s device, unless you have a clear business intent that you need to fix a problem or something. And so a lot of our financial customers love that it stays on their customer device unless they’re specifically trying to find a problem and fix it and that you can turn it off then when you don’t need it anymore. So I think from our perspective, I’d say the biggest impact will probably be the costs increases in chips and memory that being seen across the industry. I think that will just sort of drive and sort of hinder the adoption of more and more compute at the edge. I think that’s just raising costs across the board. That’s the short term cost that I see.

Swapnil Bhartiya: Thank you. Now, let’s talk about the market. First of all, how crowded is this market and how are you positioning yourselves as the pricing, availability and if possible, if you can also talk about some of your clients, you can name them, not name them. So there are two or three questions bundled together.

Peter Morelli: So generally I’ve been a buyer of software and observability software for quite a long time at many different companies and I pretty much hated every Single vendor I’ve worked with and I think it’s really due to the pricing model that they have. It’s an ingestion based model. So if you log more or you add more metrics or you, you collect more sessions, they charge you more for the privilege of doing that. And that’s true of all observability, not just mobile. The problem with that is that then you’re sort of in this weird scenario where you’re like, should I log this or not? I had whole observability teams that would go to other engineering teams and be like you’re logging too much. Which is sort of an odd incentive to sort of put into the systems.

Swapnil Bhartiya: Right.

Peter Morelli: You’re trying to fix your customers issues, why are you not doing it? Everything’s a trade off. I get that. I think what, what I really found was that it was a very antagonistic relationship with most vendors. So we designed our pricing, our architecture again we process a lot of things at the edge, allows us this flexibility. But we price not based on ingestion. So we’re not. You can log as much as you want, you can create as many metrics as you want. We’re not putting any limits on that side. What we do price is per unit. So think of active device or MAUs. So if our SDK is in an application and it connects to our SaaS or your hosted environment, that sort of counts as one. It can now connect one time, connect a thousand times. We sort of charge the same price for it. And so what that gives to buyers especially is a very predictable sort of bill. There’s no session overages, there’s no quota overages. You’re not surprised at the end of the month that something’s been blown out. So that’s a very nice sort of feature of how we price. The other one is that our incentives are very much aligned. The only time I’m going to grow is when you’re going to grow. And I think you have a better sense of how you’re going to grow and can predict that very well. So we become a very nice predictable player in your sort of. It’s a line item that’s like very nice to have as a budget owner. So that’s one of the nice things. I think generally though it also then you know, we can go to engineers and say don’t worry about logging, log everything because if you log it then it’s available right at runtime. And so again the whole point of trying to make as much of observability runtime versus compile time. This is part and parcel of that particular equation.

Swapnil Bhartiya: Peter, thank you so much for walking us through this. What you’re building is, I mean fundamentally rethinks how mobile operatively should be done and how it should work to make it not only agent accessible, but actually to make it kind of first class citizen versus a second, third or fourth class. So thank you so much. And of course all the folks who are watching it, please go check out BitDrift AI to learn more about it. And once again, Peter, thank you so much and I look forward to chat with you again.

Peter Morelli: Yes, thank you so much. Very much appreciated.