The second part is LakeFlow Pipelines, which is essentially a version of Databricks’ existing Delta Live Tables framework for implementing data transformation and ETL in either SQL or Python.

Business users can create pipelines using visual tools, but behind the scenes, Databricks automatically embeds best practices – resilience, self-repairing capabilities, lineage tracking (which shows ...

Who needs rewrites? This metadata-powered architecture fuses AI and ETL so smoothly, it turns pipelines into self-evolving engines of insight.

LakeFlow Pipelines: Simplifying and automating real-time data pipelines. Built on Databricks’ highly scalable Delta Live Tables technology, LakeFlow Pipelines allows data teams to implement data ...

However, he says that “from a bigger picture we operate in the space of data warehouse companies such Snowflake, Databricks, Microsoft Fabric who also want to build to bring AI to the enterprise.” ...