Guest: Josh Williams
Company: observIQ (Twitter)

observIQ is focused on developing fast and powerful observability technologies, including data pipeline and log management solutions. It is one of the active contributors to the OpenTelemetry project. In this episode of TFiR: Let’s Talk, Swapnil Bhartiya sits down with Head of Engineering Josh Williams to talk about how the company and its contributions to the OpenTelemetry project are helping customers.

Key highlights of this video interview:

• observIQ facilitates the collection of log traces and metrics and enables customers to send that telemetry to any platform or destination of their choice.
• Vendors had very specific APIs and very specific requirements for collecting data. With the advent of OpenTelemetry, there is a now consistent model that is being used. How data is collected and how data is ingested have been standardized. Vendors are starting to switch over to ingesting OpenTelemetry Protocol (OTLP) format.
• Early on in observIQ’s history, it was focused on facilitating the collection of data for its customers. It has evolved into taking that data and helping them reduce the cost of sending that data to the platforms. This can be done by deduplicating common logs to reduce the number of logs that are being sent. With BindPlane, it created a processor in OpenTelemetry that allows customers to take logs and convert them into metrics.
• observIQ has to look at its deployment models slightly differently when it comes to Kubernetes. There is an issue where data collectors are coming online or going offline as the Kubernetes environment is scaling, i.e., in an ephemeral state.
• OpenTelemetry is a large open-source project with a large community. It has its own challenges: 1) several different voices are pushing for different things that they would like to prioritize, 2) very complex configurations that can be intimidating to build from scratch.
• observIQ has seen customers feed their requirements into ChatGPT and successfully create quite complex configs that solve their use case. It is not necessarily a solution to issue #2, but an interesting use of technology in order to use another technology.

This summary was written by Camille Gregory.