Databricks, the company behind the Apache Spark data analytics engine, has announced that its open source Delta Lake project will become a Linux Foundation project and use an open governance model. The open governance model encourages participation and technical contribution and will also provide a framework for long-term stewardship. The Apache 2.0 software license remains unchanged though.
Since its launch in October 2017, Delta Lake has been adopted by over 4,000 organizations and processes over two exabytes of data each month.
“Bringing Delta Lake under the neutral home of the Linux Foundation will help the open source community dependent on the project develop the technology addressing how big data is stored and processed, both on-prem and in the cloud,” said Michael Dolan, VP of Strategic Programs at the Linux Foundation.
Databricks’ CEO and cofounder Ali Ghodsi, on the other hand, is confident that “Delta Lake will quickly become the standard for data storage in data lakes” through the strength of Linux Foundation community.
The company made the announcement at the Spark+ AI Summit in San Francisco.