Databricks takes on Snowflake, MongoDB with new Lakehouse Apps
Databricks on Tuesday said that developers will be able to build applications on enterprise data stored in the company’s data lakehouse and list them on the Databricks Marketplace.
Dubbed Lakehouse Apps, these new applications will run on an enterprise’s Databricks instance and use company data along with security and governance features provided by Databricks’ platform, the company said, adding that the new capabilities are aimed at reducing time and effort to adopt, integrate, and manage data for artificial intelligence use cases.
“This avoids the overhead of data movement, while taking advantage of all the security, management, and governance features of the lakehouse platform,” according to dbInsights’s principal analyst Tony Baer .
[ Also on InfoWorld: How to choose a cloud database ]
Lakehouse Apps an answer to Snowflake’s Native Application Framework?
Databricks’ new Lakehouse Apps can be seen as an answer to Snowflake’s Native Application Framework, launched last year to allow developers to build and run applications from within the Snowflake Data Cloud platform, analysts said.
Snowflake and MongoDB, according to Constellation Research’s principal analyst Doug Henschen, are also encouraging customers to think of and use their products as platforms for building applications.
“So last year Snowflake acquired Streamlit, a company that offered a framework for building data applications, and it introduced lightweight transactional capabilities, which had been a bit of a gap,” Henschen said, adding that MongoDB, which is already popular with developers, has also increased its analytical capabilities significantly.
In a move that is similar to what Snowflake has done, Databricks has partnered with several companies, such as Retool, Posit, Kumo.ai, and Lamini, to help with the development of Lakehouse Apps.
During the launch of the Native Application Framework, Snowflake had partnered with companies including CapitalOne, Informatica, and LiveRamp to develop applications for data management, cloud cost management, identity resolution and data integration.
While Databricks' partnership with Retool will enable enterprises to build and deploy internal apps powered by their data, the integration with Posit will provide data professionals with tools for data science.
“With the help of Retool, developers can assemble UIs with drag-and-drop building blocks like tables and forms and write queries to interact with data using SQL and JavaScript,” Databricks said in a statement.
The company’s partnership with Lamini will allow developers to build customized, private large language models, the company added.
Lakehouse Apps can be shared in the Marketplace
The Lakehouse Applications, just like Snowflake applications developed using the Native Application Framework, can be shared in the Databricks Marketplace.
The company has not provided details about revenue sharing or how agreements for these applications will work between two parties.
Snowflake charges 10% of the total transaction value for any applications sold through its marketplace. The company had earlier said that it would put a grading scale in place for higher value transactions.
Databricks’ new Lakehouse applications, according to Henschen, is aimed at increasing the "stickiness" of the company’s product offerings, especially at a time when most applications are driven by data and machine learning.
These new apps can be seen as a strategy to convince developers that Databricks’ platform can handle the transactional capabilities required to build a modern application, Henschen said.
The Lakehouse Apps are expected to be in preview in the coming year, the company said, adding that Databricks Marketplace will be made generally available later this month.
Databricks to share AI models in the marketplace
Databricks will also offer AI model sharing in the Databricks Marketplace in an effort to help its enterprise customers accelerate the development of AI applications and also help the model providers monetize them.
The company said that it will also curate and publish open source models across common use cases, such as instruction-following and text summarization, and optimize tuning or deploying these models on its platform.
“Databricks’ move to allow AI model sharing on the marketplace echoes what Snowflake is doing in its marketplace, which last year expanded from just data sets to include native applications and models as well,” Baer said.
Additionally, the marketplace will host new data providers including S&P Global, Experian, London Stock Exchange Group, Nasdaq, Corelogic and YipitData, the company said. Healthcare companies such as Datavant and IQVIA as well as companies dealing with geospatial data — such as Divirod, Safegraph and Accuweather —will also provide data sets on the marketplace.
Other data providers include LiveRamp, LexisNexis and ZoomInfo.
The AI model sharing capability is expected to be in preview next year.
The company also said that it was expanding its Delta Sharing partnership footprint by tying up with companies such as Dell, Twilio, Cloudflare and Oracle.
Delta Sharing is an open source protocol designed to allow users to transmit data from within Databricks to any other computing platform in a secure manner.
