StarTree, the cloud-based real-time analytics company, has introduced a new integration with Grafana, allowing users to run real-time analytics using Prometheus query language (PromQL)-compatible queries.

In this show, Peter Corless, Director of Product Marketing at StarTree, talks about the open source roots of StarTree with Apache Pinot at the heart of everything that the company builds around real-time data analytics.

Corless highlights the benefits of StarTree Cloud, including its scalability and cost efficiency, and how the free tier offering, which includes 100 gigabytes of storage, enables users to collaborate on projects and prototype real-time analytics without any upfront costs. Corless says, “It’s more than just an integration with Grafana. We’re talking about a whole new platform — a new way to be able to performantly and affordably scale your operations with StarTree, powered by Apache Pinot.”

Introduction to StarTree, Apache Pinot, and the challenges of real-time analytics

• Corless talks about the challenges of real-time analytics and observability in today’s fast-paced environment.
• Corless explains the emergence of the “impatience economy,” a landscape where users expect immediate access to data and are quick to abandon services if there’s any delay, underscoring the need for instant insights.
• Corless highlights the inherent limitations of traditional data warehouses and transactional databases, which were not designed to provide the speed and efficiency required for modern real-time analytics.
• Corless discusses how Apache Pinot, an open-source real-time analytics platform developed by LinkedIn (now managed by the Apache Software Foundation) was created to handle LinkedIn’s high-demand use cases, enabling fast, scalable data analytics.

StarTree’s managed Apache Pinot and the integration options

• StarTree’s fully managed version of Apache Pinot offers a comprehensive solution that removes the complexity of managing the infrastructure, saving businesses significant operational overhead and resources.
• Corless explains the critical role of upstream data streaming methods, such as Apache Kafka, Redpanda, and Apache Pulsar, which work seamlessly with Pinot to ensure a steady flow of real-time data for processing.
• StarTree provides multiple integration options, including support for popular Business Intelligence (BI) tools like Tableau and Apache Superset, along with embeddable integrations that allow developers to incorporate Pinot into custom mobile and web applications.
• Corless stresses the importance of maintaining high data freshness and achieving ultra-low latency when querying and visualizing data, making these features essential for businesses aiming to provide real-time insights to users.

StarTree’s involvement in Apache Pinot and the benefits of open-source projects

• Corless explains StarTree’s involvement with Apache Pinot and how the company balances open-source with commercialization efforts.
• Corless explains that large tech companies like LinkedIn, Uber, and Stripe have the internal expertise to manage their own Apache Pinot clusters. However, many businesses lack the technical resources to deploy and maintain Pinot effectively.
• StarTree steps in by offering a full service that covers everything from initial cluster sizing and specification to ongoing operations and customer success, serving as a “force multiplier” for businesses that require real-time analytics without the overhead.
• Corless highlights how open-source projects like Apache Pinot solve fundamental challenges while creating a vibrant ecosystem that fosters innovation and supports commercial use cases, ensuring businesses can build on a strong foundation.

The new Grafana integration announcement and product demo

• Corless announces StarTree’s new integration with Grafana, a leading observability tool, allowing users to leverage real-time analytics within their existing Grafana dashboards.
• Corless demonstrates how to create a new data source for Apache Pinot inside Grafana, including setting up HTTP bearer authentication and connecting to the Pinot controller URL, highlighting the simplicity of the process.
• Corless showcases the integration by running queries on complex website data, selecting time ranges, and displaying detailed graphs showing clicks, errors, and page views in real time.
• Corless demonstrates advanced multi-dimensional querying capabilities, which allow for grouping by attributes such as country, platform, and browser. Corless also walks through manual query edits to fine-tune the results.

The scalability of Apache Pinot and its impact on observability

• Corless discusses Apache Pinot’s scalability, highlighting how it can handle massive datasets that traditional time-series databases struggle with.
• Corless shares how Uber stores up to 12 petabytes of data using Apache Pinot, illustrating its suitability for industries such as IoT and mobile device monitoring, where large-scale data ingestion and querying are critical.
• The integration with Grafana supports scaling efficiently and affordably, eliminating the need for multiple observability databases by allowing all queries to run through a single, optimized platform.
• Corless emphasizes the importance of query efficiency and data storage optimization in Pinot’s design, making StarTree a valuable solution for both user-facing and customer-facing analytics at scale.

StarTree’s target audience and their evolving observability stacks

• StarTree is primarily designed as a business analytics tool, not an observability company, although its capabilities extend to observability use cases as well.
• Corless touches on the emerging trend of disaggregated observability stacks, where companies select different layers for their specific needs, rather than relying on a single, all-in-one solution.
• Corless highlights the importance of supporting various query languages such as PromQL, TraceQL, and LogQL, enabling users to perform multi-stage queries across different tables and systems, making it adaptable to evolving observability requirements.

StarTree’s role and suitability in the observability ecosystem

• StarTree’s primary focus is on the query efficiency and storage layer, offering fast data aggregations and high concurrency while minimizing resource consumption, differentiating it from traditional observability platforms.
• Corless emphasizes that while StarTree is well-suited for observability tasks, its key strength lies in powering user-facing and customer-facing analytics, especially for IoT, mobile applications, and other as-a-service solutions.
• Corless highlights how Apache Pinot’s columnar storage architecture provides significant cost savings and real-time results, making StarTree a strategic choice for businesses needing large-scale observability solutions alongside fast analytics.

StarTree’s Pricing model, trial opportunities, and the benefits

• StarTree offers a free tier, which includes 100 gigabytes of storage, allowing users to collaborate on projects and prototype real-time analytics without any upfront costs.
• Corless details how the serverless platform enables users to easily roll out an Apache Pinot cluster in minutes and, once ready, deploy it to production with minimal effort.
• Corless encourages listeners to visit StarTree’s website where they can sign up for free and explore the full capabilities of the platform, emphasizing its ease of use and cost-effectiveness for businesses of all sizes.

This summary was written by Emily Nicholls.