observIQ focuses on observability telemetry, building agents, integrations, and more to gather, process, send, and transmit the data through an observability pipeline. BindPlane is their first open source product which aims to help manage the open source agents, in particular, the OpenTelemetry Collector, to make it easy to configure, update, and manage a full fleet of observability telemetry. One of the key challenges in large environments is managing those agents across hundreds or thousands of hosts. In order to manage the massive amount of telemetry data, you need a solution layer on top like BindPlane.

The company has partnered with Google for a number of years to expand the visibility you get from the Google operations platform, not just the Google Cloud components and services but also for third-party applications.

In this episode of TFiR Let’s Talk, Swapnil Bhartiya sits down with Mike Kelly, CEO of observIQ, to discuss  the company’s focus in the telemetry space and their open source product BindPlane. He also takes a deep dive into their partnership with Google Cloud and the challenges people are having with managing the flow of data.

Key highlights of this video interview are:

• observIQ is focused on observability telemetry, building the agents, integrations and everything else in order to gather, process, send, and transmit the data through an observability pipeline. Kelly explains the trends they are seeing in the space as it moves to open source and standardization of telemetry.
• Kelly discusses one of their key partnerships, Google, and how they are supporting their customers in the field and accelerating the expansion of the Google operations platform. He goes on to tell us about BindPlane, their first open source product, and how it is making it easy to configure, update, and manage your full fleet of observability telemetry.
• observIQ has been working with Google for many years expanding the visibility that you get from Google operations platform. One of the key challenges customers experience when migrating from one place to another is having centralized visibility. Kelly takes a deep dive into how their partnership helps expand that visibility and how it benefits the community too.
• Kelly explains how BindPlane gathers and processes the data. He describes the challenges of managing agents across hundreds or thousands of hosts and why you need a solution layered on top of that. He goes into detail about the different versions, sharing details about some use cases where BindPlane is needed and how it helps.
• Since the quantity of data has grown, so have the challenges of managing it along with the cost of storing and processing it. Kelly believes that this evolution has driven the need for an observability pipeline layer to inject some processing capability before it gets to the final destination. He explains how observability pipelines are helping to tackle these problems while reducing costs and improving visibility.
• The primary goal of observability solutions from a performance management perspective is managing and understanding your uptime. Kelly believes it is a critical piece of the business delivery. He continues explaining where people can go wrong with observability, such as not being able to quickly identify if there is a failure and resolve it.
• BindPlane went into beta phase in June and has so far received positive feedback. Kelly goes into detail about the roadmap to expanding it into a full-blown observability pipeline solution. He explains that they expect it to be generally available in August and discusses what else to expect from their offerings.

Connect with Mike Kelly (LinkedIn)

Learn more about observIQ (Twitter)

The summary of the show is written by Emily Nicholls.

[expander_maker]

Here is the automated and unedited transcript of the recording. Please note that the transcript has not been edited or reviewed. 

Swapnil Bhartiya  0:00 This is your host, Swapnil Bhartiya. And welcome to another episode of TFiR Let’s Talk. And today we have with us once again, Mike Kelly, CEO of observIQ. Mike it’s great to have you back on the show again after a long time.

Mike Kelly  0:12  Yeah, it’s great to talk with you again, been fortunate.

Swapnil Bhartiya  0:16  Yeah, and I, if I’m not wrong last time, when I talked to you, your CTO and all your CEOs, I also understand, you know, I want to revisit recap a few things, first of all about observIQ, and that in your role as CEO, how do you look at the check challenges in this space that you folks are trying to solve? And how are you going to tackle them in your new prediction?

Mike Kelly  0:35  Yeah, I appreciate that. So as you mentioned, I shifted from CTO and observIQ to CEO earlier this year. And, you know, it’s an exciting time for the company and what we’re, I think, for space as well. And observIQ, we’re really focused on observability telemetry, so building the agents, and the integrations and the observability, pipelines, really everything that you need to, to gather processes, and send and transmit the data through an observability pipeline. And doing that through Open Source. So, you know, with projects like open telemetry, and others, right now moving the space to Open Source and to standards, as has been, I think, a, you know, an incredible time for the industry, and particularly for the standardization on telemetry. So, you know, as we’ve been focused on that, over the last several years at observIQ, we’re starting to see that accelerate with new products, and just an acceleration of the development of these Open Source projects that are already out there.

Swapnil Bhartiya  1:40  Excellent. Since you mentioned Open Source projects, you know, of course, OpenTelemetry, formed through a merger of the OpenTracing and OpenCensus projects. I also want to understand the right use of involvement with some of these projects, it could either be consuming them or, you know, offering commercial support with them or contributing to them. So there are three C’s.

Mike Kelly  2:05  Yeah, a great question. So when it comes to these Open Source projects, we’ve been partnering with vendors to work on them together, and then provide them to customers. So we have a partnership that we’ve had with Google and have had for a number of years, and they’re one of the primary folks that we’re working with, you know, to support their their customers in the field and accelerates the expansion of the Google ops, Google operations platform and to other third party applications and on prem support and support for other clouds. And doing that through Open Source. So that’s, you know, one of the ways that we’re involved, we also develop and distribute our own products that are built around that. And most recently, we’ve just launched into beta, a product called BindPlane. And this is the first Open Source product that’s designed to manage these Open Source OpenTelemetry Collector, and manage the configuration, the pipeline management, and make it really easy to configure, update, and manage all of your full fleet of observability telemetry.

Swapnil Bhartiya  3:21  Excellent. Let’s go a bit deeper into both of these. But let’s start with the Google Cloud Platform partnership, talk about, you know, a bit about this partnership, or what do you folks are doing together? And you know, how you’re helping joint customers?

Mike Kelly  3:34  Yeah, great question. So we’ve been working with Google for many years now. And the partnership plan has been focused on, you know, expanding the visibility that you get from Google operations platform. So the ability to use their solutions to monitor not just your Google, Google Cloud components and services, but also to use that for third party applications. You know, custom logs, if you’re trying to gather, too, if you want to pull in data from other cloud vendors, you can do that as well. And even on prem data center, that’s one of the challenges that a lot of customers have is as you’re migrating from one place to another, you really need to have centralized visibility, even if you have, you may still have a homegrown data center that you’re managing. So what we do is partner to expand that visibility. And we do that a couple of ways. The most relevant one today is by working jointly on some of these projects, and expanding the visibility. So for example, this year we’ve developed a number of metric integrations, log integrations and also a very high performance log agent that’s designed to process data really easily. It’s highly configurable and something that is, you know, Open Source and available for, you know, very large enterprise customers. And ultimately we decided to, to donate that to the open telemetry project, and have been working jointly to incorporate that into the project so that it’s available as a piece of open telemetry, which, which we think is really exciting to have everything from traces, but also now metrics and, and finally logs all within one collector that can be really universal. And when you think of that, then then you know that that’s, as I mentioned, something we’ve worked with Google on. But also it really benefits the entire community because this is a project that is becoming a one stop shop for all of your telemetry needs. And you can use this as a really the universal solution for gathering and processing telemetry data. So we find that exciting. I think that there’s a great future ahead for this and other projects like it.

Swapnil Bhartiya  5:57  Yeah, thanks for explaining the kind of partnership you folks have with Google. Now Let’s Talk about BindPlane. What is it all about? And also, once again, does it serve the same common audience? And if yes, how does it complement the effort that you’re making with other players?

Mike Kelly  6:14  Yeah, great question. And there are a couple of things that we, you know, when we’re talking about buying plane tickets I like to focus on one is the challenges of, of telemetry. So we talked a little bit about the collector. And that’s the first piece right, you need to you need a way to instrument and gather the data and process the data. But in very large environments, it becomes challenging just to manage those, those agents across hundreds or 1000s of hosts. So when you’re really trying to trying to manage massive amounts of telemetry data, you need a solution layered on top of that. And that’s where BindPlane comes in. And BindPlane is a product that we’ve had for a number of years. And prior to this latest release, it was always a SaaS only offering. And so what we’re doing today is we’re providing this as a distribution, it’s Open Source, we’re open sourcing the product. And we’re also providing a version that you can deploy within your own environments, that’s been a big need. And something that people are looking for. What it lets you do is control all of your agents, deploy agents, really easily manage the configuration across a large fleet of hundreds or 1000s of agents, and control the flow of data to multiple destinations. So you can think of some use cases like, you know, maybe you have a fleet of hosts that you’re trying to monitor, and, and it’s a few 1000 of these. And maybe some of them are within a certain group development group. And you want to apply the same type of configuration to those BindPlane makes that really easy. And it’s rather than being a gent general, or generic type of a DevOps tool, it’s really dedicated to telemetry, and that makes it easier to make adjustments. So if you, for example, decide that tomorrow, we want to reduce the amount of log data flowing through my system, you can make those adjustments within BindPlane, or if you decide, well, I want to send off compliance data compliance logs and send that to a low cost storage bucket somewhere, you can do that with find payment. And it doesn’t require RE instrumentation and going through every single host or agent. And either deploying new agents, deploying new configurations, all the complexities that you have, if you’ve done this before, you know it can be an incredibly challenging task. So the solution here is to simplify the deployment, simplify the configuration, simplify the management, and then simplify how you get data from source to destination.

Swapnil Bhartiya  8:53  Now, let’s assume out of you know, observIQ and the work that you folks are doing and just look at this observed, the opposite will be the space itself. First of all, if you can, please, you know, because last time it will take it’s been wild, how the observer is pace has evolved, because the workloads attending the deployment of Kubernetes has increased. A lot of folks are putting it in production, which also means that the challenges are different now. There is more awareness about it as well. So from your perspective, how do you see the evolution of this space?

Mike Kelly  9:25  Yeah, great question. And, you know, we have an I certainly have a biased view on that, as does everyone right. But I think one of the areas that you see a lot of the changes is just the quantity of data, right that that’s available and is required. And as the quantity of data has grown, the challenges of managing has grown, the cost of of storing and processing it has also grown and you you see this new emerging need or at least acknowledgement of The need for an observability pipeline layer, so some type of way to, to inject some processing capability before it gets to that final destination. And that does a lot of things, right? It simplifies the flow of data, it allows you to, to send data where it needs to go and allows you to reduce the amount of data that’s flowing. So if you’re, what you’ll find in large environments is you end up with this, you know, it’s you end up collecting all the data, all the logs, because it’s just too difficult to really narrow down what you need. And observability pipelines help you avoid that, right, because they make it easier to go in and actually manage the data as it’s flowing through the pipe, reduce the data, filter down the data, and ultimately use some AI and ML on that data prior to getting to that destination. And when you really when you layer this in between it solves a lot of problems, and it opens up a lot more opportunities. And some of those are the use cases, I think we’ve talked a little about whether it’s compliance, a lot of companies are struggling with compliance, they need low cost storage for all of these logs, they don’t necessarily need it in a in a high cost sim or, you know, performance management solution. So with that with a pipeline that makes it easy to collect and send that data to your low cost storage. And the same way, you can use that same instrumentation to send it wherever else it needs to go. So over the last few years, just seeing that evolution, and this is something that’s recognized as a solution that, you know, unlocks a lot of potential can reduce your costs, but also just improves, improves your visibility and kind of keeps your eye on what’s important. That’s been pretty notable to me

Swapnil Bhartiya  12:00  You know, when you and I sit down, we love to talk about technology. But the fact is, when we talk about businesses, enterprise customers, of course, they do use technology as a means to an end. But they are solving a specific problem for the customers, they are solving a specific problem for business continuity, or adding new features and value. So when we look at observability, can you talk about? Where does it fit into the whole business landscape? You know, so that, you know, they do when they’re, you know, Client Success teams? They look at it, you know, very much like that? Yes. observability is a critical piece of our business delivery?

Mike Kelly  12:35  Yeah, that’s a good question. And, of course, you know, as people are focused on technology, it’s easy to gloss over. Right. But ultimately, the primary goal of observability solutions, particularly if you’re looking at performance management, is managing and understanding your uptime, right? Are these systems available? Are they delivering? And can your business continue at the pace that it’s at? And so if you’re not achieving that goal, then it’s been I think that it’s fair to say that observability at least in this case, is not achieving its goal? So, you know, I think that it’s a critical piece. And there are a couple of ways that I think people their missteps along the way, right? I think that sometimes what you’ll see in observability, is, you know, you take your eye off the ball, we really want to focus on making sure that things are operating correctly for the end user and making sure that this is successful and having a tight loop. So as there’s an issue, how do we resolve that quickly? And if you get into a place of gathering everything, right? If it’s for the sake of observing everything, you really can’t do that effectively. So you know, there’s been a lot written on this, whether it’s around SRE, some good work around SRE that, you know, Google has done focusing on the key signals that really matter. But ultimately, that’s, that’s where, where you really need to rely on observability. And without it, you’re lost in a lot of ways. If you don’t have solutions in place, that are helping you quickly identify if there’s a failure, effectively do that. And also iterate quickly. It’s really difficult to hit the SLO’s that you’re committing yourself to. So you know, certainly a key piece of that. And also a piece that’s really, really easy to get wrong, or to overdo sometimes.

Swapnil Bhartiya  14:28  Excellent. Thank you for actually taking that question. Because I’m happy that we were able to add that. Now. Let’s go back to the project. Earlier, we were talking about BindPlane, it’s in beta. Can you talk about first of all, what kind of time frame is an Open Source project? So especially Open Source, I don’t have a specific timeframe. It’s beta. But you know, it’s in the beta phase. So what is your kind of, you know, plan for this beta, how long will you run the beta? Who can access it? How can They can access it and what kind of feedback are you expecting? And when can we see the GA for this project?

Mike Kelly  15:07  Yeah, thanks for asking about that. So it’s in beta and went into beta in the middle of June. So it’s been out for a little under a month. So far, we’ve had great feedback. So far, it’s publicly available. So if you go to .bserviq.com, you’ll find plain op. So go in and download it and try it out. And really what we’re looking for right now in the space is, there a lot of there’s a lot of functionality on the roadmap and expanding this into a, you know, full blown observability pipeline solution is, is the goal that we want to know what people are looking for today, and what they want to see in the next couple of months and through the end of the year. So we expect this to be in GA in August, so it’s coming up really quickly. And there’ll be additional SaaS versions that will be provided later this year. So a pretty aggressive roadmap. There’s a lot there. And I think even today and beta, it’s a really compelling product that love it, if more people wanted to give it a shot. We also have a Slack community. If you’re interested in trying it out, you can find all that at observiq.com

Swapnil Bhartiya  16:16  Mike, thank you so much for taking time out today and talk not only about your partnership with Google Cloud Platform, announcement of beta for BindPlane but also in general, where observability spaces work enough evolution we are interesting there and also what to expect in the coming months. We cannot talk about two years at this point. So thanks for sharing all those insights and I would love to have you back on the show as usual. Thank you.

Mike Kelly  16:39  Excellent. I appreciate it was great to talk to you again.

[/expander_maker]