The second part is LakeFlow Pipelines, which is essentially a version of Databricks’ existing Delta Live Tables framework for implementing data transformation and ETL in either SQL or Python.

LakeFlow Pipelines: Simplifying and automating real-time data pipelines. Built on Databricks’ highly scalable Delta Live Tables technology, LakeFlow Pipelines allows data teams to implement data ...

Who needs rewrites? This metadata-powered architecture fuses AI and ETL so smoothly, it turns pipelines into self-evolving engines of insight.

Business users can create pipelines using visual tools, but behind the scenes, Databricks automatically embeds best practices – resilience, self-repairing capabilities, lineage tracking (which shows ...

However, he says that “from a bigger picture we operate in the space of data warehouse companies such Snowflake, Databricks, Microsoft Fabric who also want to build to bring AI to the enterprise.” ...