OpenSearch as AI Data Infrastructure | Bianca Lewis, OpenSearch Software Foundation | TFiR

Swapnil Bhartiya: Some five years ago, a license change logged teams out of the very software stack they helped build and maintain. And the whole team responded the way in general we see response from Open source. They rebuilt on open ground and promised to not get trapped again. That was the birth of OpenSearch. Then the OpenSearch moved to the OpenSearch Software Foundation. And recently the community behind OpenSearch, the OpenSearch Software foundation, marked five years with the new data infra report, new members, more numbers and of course Open sourcecon as well. And today we have with us once again Bianca Lewis, executive director of the foundation, to talk about these five years where OpenSearch is heading and this event. First of all, Bianca, it’s great to have you on the show.

Bianca Lewis: So Swapnil, great to be on the show and always a pleasure to speak with you. And as you said, five years, time flies when you’re having fun. It’s unbelievable.

Swapnil Bhartiya: Yeah, it’s so true. I remember the first time when it came out and we talked about it and I look at it’s always five years. Wow. Let’s start with OpenSearchCon. Of course it’s five years project has some names like Apple, Cern, LinkedIn and they were on stage as well at San Jose. So talk a bit about first of all, those who may not know, let’s lay the foundation for them. What is OpenSearch all about? And then we’ll talk about this event. How was the mood at the event? And from your perspective, where do you see the project stands in the five years? So there are three questions bundled, but the aim is the same.

Bianca Lewis: Okay, so why don’t we start right at the beginning? And Open Search Con North America was amazing, both in terms of itinerary, in terms of companies that were on the stage giving keynotes, but also in terms of attendees. And what is strange, I remember going to the very first OpenSearch con five years ago, it was in Seattle and I would say 80, 90% of those first attendees are still attending today. And that’s incredibly special to have such a loyal community plus all the new blood joining us. And it’s not only that we turned five years today, we got amazing momentum, which we’re going to go into soon. But in addition to that, we also requisitioned for our five year birthday the largest research project on data infrastructure ever commissioned by OpenSearch that was performed by the Linux foundation and not just amongst OpenSearch users. So the amount of data, the amount of momentum, key talks, conferences, birthday parties, it’s just been such a wave of enthusiasm that speaking now is a great time to chat.

Swapnil Bhartiya: You folks also made some news there, of course, new gen members, including Intel Certified Long term support. This is just. I’m skipping the service. You just. You can walk us through some of the major announcements that you made at this event.

Bianca Lewis: Yes, so we announced membership growth, which is three new members. Like you said, it was Intel, Adelean and Sidecar. We also announced Seacom and Sidecar as new LTS vendors. If we go to previous announcements, you’ll see that was announced in April in Prague to give companies accredited official support of OpenSearch should they wish it. In addition to that, what we released is we released 3.8. We also released the results of the research project, which tells a really interesting story about the growth of OpenSearch to the platform that it is today and the metrics that we’ve achieved today as well. And I think on that journey from your question is how did we get from there to here? And as you said, it was a fork when there was a license change and the community really wanted an open platform and it was driven by AWS at the time with the contributions of a few smaller companies. The intention from day one was always an open platform. But I think with all of those license changes that you refer to Swapnil, once you betray an open source community, it’s very tough to get trust back. And what AWS did is after it gave it a really strong foundation as a platform, it donated it to the Linux foundation two years back. And once they did that provided with credibility on the vendor neutral home where no single vendor could dominate OpenSearch. And with that credibility we’ve grown from about 700 million downloads to 2.4 billion downloads with over 400 companies contributing actively with almost 3,000 contributors. But it’s more than that because the contributor diversity has grown by 31% in the last year alone. The awareness of OpenSearch from two years ago has grown from 68% to almost 90%. Most companies. So very interesting, very interesting in terms of momentum, in terms of new growth and new announcements and of course some new plans.

Swapnil Bhartiya: I talk about plans, I just want to talk about these numbers. As you rightly mentioned, it’s almost like 140% in a year. And the contributions, you know, contributors also grew almost 31% as you rightly mentioned. Of course, neutral umbrella of Linux foundation backing by some of the biggest players that do give some confidence. But what is fueling that momentum? Because also OpenSearch came at the right time with the whole suddenly the explosion of AI as well. So it was in the right place at the right time and under the right umbrella as well. So talk about what is fueling this momentum.

Bianca Lewis: That’s an interesting question which was I think largely answered by the research project that we just mentioned now is that I think that is really fueled by the adoption of AI workloads. Now, about a year ago we started to speak in different terms. We started to speak not in terms of platform use cases, not an observability use case and a search use case and a security use case and a business intelligence use case or an APM use case. We started to talk about at AI scale, we executing hundreds of thousands of queries a second. Working in those silos is a massive data tax security risk and unmanageable on scale and economics. We consolidated the position of OpenSearch as an AI data infrastructure layer. I think that that message directly impacted this wave of growth now because what the research found was that 83% of all organizations that were surveyed, and that was almost 300 different organizations, are either running or plan to run AI workloads. That figure goes to more than 90% if you go to large enterprises. Now that by definition is going to require that data infrastructure layer, which is feeding into exactly what OpenSearch does when companies making decisions of what platform can we unify our AI workloads on. It was really interesting to hear that 78% of organizations were evaluating data infrastructure tools based on flexibility, where one tool can do more than one thing, so they can unify these use cases together as the messaging goes. But not far behind that there were the usual suspects, right? Total cost of ownership, again, the open platform, which is driving that almost 80%. It was really important to not have any vendor lock in or independent of every cloud provider. And this was driven as a strategic decision and priority, not just a want to have. So all of this is really fueling into the growth of OpenSearch today, which positions us nicely, but it also positions OpenSearch with great responsibility that we make the project innovative and sustainable for the community to use and enjoy.

Swapnil Bhartiya: And since you quoted of course the research, you know, of course the Linux foundation research just released the 2026 Open Inflation Report as well. Talk a bit about what was the most interesting trends that you noticed in that report that even you were like, wow, this is exciting.

Bianca Lewis: What I found really exciting and we spoke about this, that the awareness of open search is close to 90% today. However, production adoption has increased from around 19% to 36% of workloads into production, it’s nearly doubled. But the amazing figure about that Swapnil is that the amount of OpenSearch clusters in production are matched by the amount of open source clusters who are still in POC or testing phases or non production phases, which was not the case just a couple of years ago. So the growth has been massive, but the pipeline of growth coming seems to be just as big. So I think the future is extremely exciting and extremely bright. And in the survey it actually highlighted that 81% of respondents who are today not using OpenSearch are really happy and willing and want to evaluate OpenSearch for the AI data infrastructure needs. So as much momentum as OpenSearch has got, I think that the real innovation in golden age is still ahead of us.

Swapnil Bhartiya: When it comes to data. Because apps can come and go. Data is something which is really, really critical to organizations. The report also found that 71% of organizations now treat deploying independently of any single vendor because that does raise the risk. Of course, vendor lock in it is becoming a strategic priority. Can you talk about why has vendor neutrality became such a non negotiable topic for data infrastructure? And what role is OpenSearch playing in enabling that?

Bianca Lewis: I do believe that that’s very closely linked to the AI discussion that we just had from different perspectives on an AI scale. A lot of companies today are not really sure how much AI workloads are going to cost them. We all heard the token economy and the buzz around that. But the fact is how much do my applications really cost me at AI scale? And if we find ourselves losing money on an application, what does that mean for the business? How do we secure things at scale as well? What happens with all of these increase in workloads that we might expect or might not expect and a vendor suddenly raises their prices or gives a pricing model, then I have to re architect my whole data infrastructure. What about all the compliance like CRA and data sovereignty that are coming up here? And the challenges are very numerous if you put in your eggs into a proprietary vendor or a single cloud. And that trend has become very clear, that the safest progress in this present time of uncertainty is to develop on an open infrastructure where you own your own data infrastructure in a sovereign way. When you figure out all of these questions and what they mean to your business, your choices will become clearer in one year, two years, three years. But the priority at the moment is not to lock into any one approach. And there’s no better system for that than the open platform of OpenSearch.

Swapnil Bhartiya: We have talked about the community, of course, we talk about the holy co. Let’s talk about the software itself. The latest release was of course it landed right around the conference. Talk a bit of what’s new in this release.

Bianca Lewis: Yeah, I think that in the latest release, and the latest releases are about real world improvements which are iterative about new capabilities. It’s not about performing 0.3 milliseconds in this use case at midnight, it’s about practically getting us further along our AI journeys on the data infrastructure. So some concrete examples of what I’m talking about here, okay, is that we’ve extended our model context protocol integrations to even more agent types. More than that, we ingest vectors now at 4.16 times faster than the previous version and improving radial search throughput by more than double. In terms of then we can of course scale the relevance evaluation with the access to LLMs and providers. Whether you use open weight or APIs, we’re agnostic to the LLM models that you actually using. And then of course we’ve streamlined log analysis by using a visual PPL builder which is now in OpenSearch out of the box, which includes SQL queries and an onboarding canvas for you to use and things like this, which actually makes it pretty easy to compare things like time series data and PPL commands and things like this. So the real world application on scaling, vector search, smart search, hybrid search, relevance training, log analysis have all scaled way beyond just a couple of versions back. And that modernization is incredibly important because as we know, software platforms were built and designed before the advent of this generation of AI. AI is not new, it’s been around for a long time. But this generation of AI, so these iterative changes of ingesting vectors four times faster with base 64 encoding, things like that, and these are really important to reach the scales that we need to deliver real world applications.

Swapnil Bhartiya: Bianca, thank you so much for first of all taking time out to talk to us and also for leading this organization and I mean this is a great example of how open source always finds a way, no matter what happens, it’s always a positive sum game. Thank you so much for joining us and I look forward to chat with you again. Thank you.

Bianca Lewis: Thank you very, very much. Always a pleasure. Thank you. Bye.