Decodable, the stream processing company, has expanded the enterprise features of its platform with the addition of a Snowflake Streaming API connector and a dbt adapter. For data engineers, this means being able to ingest real-time data into their Snowflake cloud data warehouse, taking advantage of lower cost and reduced latency, without having to integrate with Snowflake themselves. For analytics engineers, data analysts and data scientists, it is now easier than ever to manage streaming and batch transformation simultaneously in a single workflow with dbt and Decodable. Together, these enhancements advance Decodable’s mission to make stream processing simple so that every organization can successfully realize the benefits of real-time data.

Decodable’s new Snowflake Snowpipe Streaming API connector enables ingestion of real-time, streaming data directly from Apache Flink into the Snowflake cloud data warehouse via the Decodable platform—no time-consuming integration required.

Prior to the availability of Decodable’s Snowflake Snowpipe Streaming API connector, the workflow required to get streaming data into and out of Snowflake was complex, with a significant amount of ongoing overhead and labor required to maintain it. Now, Decodable makes it simple to integrate real-time stream processing with Snowflake-based data stacks. The Decodable Snowflake Snowpipe Streaming API connector is fully hands-off: all that is required of a data engineer is simply to configure the sink in Decodable.

In addition, the Decodable Snowflake Snowpipe Streaming API connector enables data scientists to to balance data ingestion latency against the cost of Snowflake warehouse usage. The streaming feature is built to handle real-time latency in a cost-effective manner, so there’s no tradeoff to be made.

Decodable’s Snowflake Snowpipe Streaming API connector is available today as a technology preview. Decodable will make the connector generally available (GA) in tandem with Snowflake’s GA of its Streaming API (projected for June 2023).

Decodable’s new dbt adapter brings the workflow and collaboration features of dbt to users of the Decodable Apache Flink-powered stream processing platform.

Decodable is releasing its dbt adapter as open source, available for contributions and feedback from the community.