The Linux Foundation Projects
Delta Lake

Delta Lake Videos

Watch the latest videos and webinars for the open-source Delta Lake project.

Delta Lake Shorts

How did Andy Grove get his start in programming? Unleashing the Power of Delta Lake Revolutionizing Data Storage in the Cloud Accelerating Data Engineering with Software Engineering Principles Unlocking Maximum Performance for Seamless Integration | Rust vs Python Unleashing the Power of DataFusion | Building NextGen Systems Rust Ensures Zero Memory Bugs No More Debugging Nightmares Unlocking the Power of Rust | No More Memory Issues Unleashing the Power of Ballista from SQL Parser to In-Memory Query Engine What led Andy Grove to create Ray SQL? #deltalake #ballista #datafusion The Rise of Rust & its Capabilities #deltalake #rustlang #rust Delta Lake - Things you need to know - 1. What is it? What is Delta Lake? #shorts #databricks #deltalake #spark #dataengineering Delta Lake Rant #shorts #dataengineering #programming #databricks #bigdata How HSBC Revolutionizes Its Data Management with Delta Tables and Streaming Florian Valeye Describes Delta Lake in The New Stack Interview Why is Rust so Important to Delta Lake? Answered by Florian Valeye How does Florian Valeye see Delta Lake maturing? How did Florian Valeye get involved with Delta Lake? The Two-Tier Data Architecture Can Be Problematic

Delta Lake Deep Dives

šŸŒŠ Let's deep dive into a variety of Delta Lake topics!

Delta Lake Deep Dive: DeltaTorch Delta Lake Deep Dive: Dagster and Delta Lake Delta Lake Deep Dive: Liquid Clustering Delta Lake Deep Dive: Rust Crate Delta Lake Deep Dive: Streaming Delta Lake with Apache Spark Structured Streaming Beyond Partitioning: Using Liquid Clustering to Escape from Hive-Style Partitions Ask the Authors Rusty Waters: Elevating Lakehouses Beyond Spark Revolutionizing Delta Lake workflows on AWS Lambda with Polars, DuckDB, Daft & Rust

Delta Rust

Rusty Waters: Elevating Lakehouses Beyond Spark Streaming Data into Delta Lake with Rust and Kafka Ensuring Correct Distributed Writes to Delta Lake in Rust with Formal Verification Tech Talk | Diving into Delta-rs: kafka-delta-ingest Upgrading from legacy to the cloud with Scribd - Part Three Upgrading from legacy to the cloud with Scribd - Part Two Upgrading from legacy to the cloud with Scribd - Part One Growing the Delta Ecosystem to Rust and Python with Delta-RS AMA | Growing the Delta Lake ecosystem Data Collab Lab: How Scribd Uses Delta Lake to Enable the World's Largest Digital Library

Delta Lake Community Meetups

Delta Lake Community Meetup (10-25-23) Delta Lake Community Meetup (02-07-24) Delta Lake Community Meetup (03-27-2024)

Data+AI Summit 2023

Delta Kernel: Simplifying Building Connectors for Delta Processing Delta Lake Tables on AWS Using AWS Glue, Amazon Athena, and Amazon Redshift Nebula: The Journey of Scaling Instacartā€™s Data Pipelines with Apache Sparkā„¢ and Lakehouse Introducing Universal Format: Iceberg and Hudi Support in Delta Lake Writing Data-Sharing Apps Using Node.js and Delta Sharing Databricks and Delta Lake: Lessons Learned from Building Akamai's Web Security Analytics Product Delta Lake AMA

Last Week in a Byte

Get the latest Delta Lake news and events, a week too late!

Last Week in a Byte (2023-05-09) Last Week in a Byte (2023-04-18) Last Week in a Byte (2023-04-11) Last Week in a Byte (2023-03-28) Last Week in a Byte (2023-03-21) Last Week in a Byte (2023-03-14) - Pi day! Last Week in a Byte (2023-01-10) Last Week in a Byte (2022-12-20) Last Week in a Byte (2022-12-13) Last Week in a Byte (2022-12-06) Last Week in a Byte (2023-05-30) Last Week in a Byte (2023-06-06) Last Week in a Byte (2023-07-18)

Conference Talks

Delta Lake 2.0 Overview Presto Tech Talk: Diving into the Delta Lake Connector for Presto Delta Lake Connector for Presto - Denny Lee, Databricks Powering Interactive BI Analytics with Presto and Delta Lake Optimizing Delta Parquet Data Lakes for Apache Spark - Matthew Powers (Prognos) Delta Lake Michael Armbrust Keynote Data + AI Summit 2022 Deep-Dive into Delta Lake Build an Enterprise Lakehouse for Free with Trino and Delta Lake Building Reliable Lakehouses with Delta Lake The Pace of Innovation in Delta Lake - Vini Jaiswal, Delta Lake Modernize Regulatory Reporting: Get Ready for T+1 Settlement

Under the Sediments: Diving into Delta Lake

Tech Talk | Diving into Delta Lake Part 3: How do DELETE, UPDATE, and MERGE work Tech Talk | Diving into Delta Lake Part 2: Enforcing and Evolving the Schema Tech Talk | Diving into Delta Lake Part 1: Unpacking the Transaction Log

Delta Lake DW Techniques

Generating Surrogate Keys for your Data Lakehouse with Spark SQL and Delta Lake Tech Chat | Slowly Changing Dimensions (SCD) Type 2 Tech Talk | Using Delta as a Change Data Capture Source

Getting Started with Delta Lake

Delta Lake Tips, Tricks, and Best Practices Making Apache Sparkā„¢ Better with Delta Lake Simplify and Scale Data Engineering Pipelines with Delta Lake Beyond Lambda: Introducing Delta Architecture Getting Data Ready for Data Science with Delta Lake and MLflow The Genesis of Delta Lake - An Interview with Burak Yavuz

Simon & Denny Ask Us Anything

Simon + Denny Ask Us Anything - December 13, 2022 Simon + Denny Ask Us Anything - November 15, 2022 Simon + Denny AUA: Data Mesh (2022-10-18) Simon + Denny AUA (2022-09-06) Simon Whiteley + Denny Lee Live Ask Me Anything Tech Talk | Simon and Denny Build a Thing! Part Two Tech Talk | Simon and Denny Build a Thing! Part One Tech Chat: BI to Lakehouse Round 3: Community Questions Answered Tech Chat: BI to Lakehouse Round 2 - Electric Boogaloo Tech Chat: Simon and Denny Discuss SQL Server and BI to Delta Lake and Lakehouses

Delta Lake Discussions with Denny Lee (D3L2)

D3L2: Let's connect: Power BI to Delta Lake / VSCode Databricks Power Tools D3L2: Implementing a Data Lakehouse for Improved Data Science and Analytics D3L2: Massive Data Processing in Adobe Experience Platform Using Delta Lake D3L2: The Journey Unifying Data Lake and Data Warehouse with Robert Kossendey at Claimsforce D3L2: The Inception of Delta Rust D3L2: Migrating from a Data Warehouse to a Lakehouse with Structured Streaming and Delta Lake D3L2: The Genesis of Delta Rust with QP Hou D3L2: Discussing Data Integration with Airbyte and Delta Lake D3L2: delta-rs at Back Market: Python and Rust, the best of both worlds D3L2: Why did we migrate to a Data Lakehouse on Delta Lake for T-Mobile D3L2: Cybersecurity, Data Mesh, and Delta Lake at HSBC D3L2: SageMaker Studio for Machine Learning on Delta Lake D3L2: Discussing Rust, Ballista, Ray SQL, Data Fusion with Andy Grove D3L2: How BASF achieves global sustainability with Delta Lake w/ Robert Pack

Delta Lake Tutorials

Module 3: Delta Lake 1.2 Tutorial with Jacek Laskowski Module 2: Delta Lake 1.2 Tutorial with Jacek Laskowski Module 1: Delta Lake 1.2 Tutorial with Jacek Laskowski (2022-05-19) What, How and when to Delta Lake - A Live Coding Session with Jacek Laskowski Tutorial: How Delta Lake Supercharges Data Lakes Delta from a Data Engineer's Perspective

Delta Lake Tech Talks

Delta Lake for Features Stores with Hopsworks Get Started with Processing Delta Tables Using AWS Glue, Amazon Athena, and Amazon Redshift Tech Talk: Serverless CDC on GCP using Datastream and Delta Lake on Google Cloud Architecting for Data Quality in the Lakehouse with Delta Lake and PySpark Level Up Your Lakehouse with Data Source Control & Cross Collection Consistency Extending Delta Sharing for Azure and Google Cloud Platform Why Data is Eating the Universe: Hosted by Seattle Spark+AI Meetup Tech Talk | Diving into Delta-rs: kafka-delta-ingest Delta Lake Roadmap 2021 H2: Features Overview by Vini and Denny Multi-Table Transactions with LakeFS and Delta Lake Upgrading from legacy to the cloud with Scribd - Part Three Upgrading from legacy to the cloud with Scribd - Part Two Upgrading from legacy to the cloud with Scribd - Part One Online Meetup | From Tahoe to Delta Lake Tech Talk Series Part Four: Continuous Integration and Continuous Delivery with Delta Lake Tech Talk Series Part Two: Boost Delta Lake Performance with Data Skipping and Z-Order Seattle Spark + AI Meetup: How Apache Sparkā„¢ 3.0 and Delta Lake Enhance Data Lake Reliability Tech Talk Series Part One: Engagement Activity Delta Lake Tempo: Distributed Time Series Analysis with Apache Sparkā„¢ and Delta Lake Data Collab Lab | Building Delta Lake with Images on Lakehouse

Delta Lake Community Office Hours

Delta Lake Community Office Hours: January 12, 2023 Delta Lake Community Office Hours: December 15, 2022 Delta Lake Community Office Hours (2022-10-20) Delta Lake Community Office Hours (2022-10-06) Delta Lake Community Office Hours (2022-10-06) Delta Lake Community Office Hours (2022-09-22) Delta Lake Community Office Hours (2022-09-08) Delta Lake Community Office Hours (2022-08-18) Delta Lake Community Office Hours (2022-06-09) Delta Lake Community Office Hours (2022-05-26) Delta Lake Community Office Hours (2022-05-12) Delta Lake Community Office Hours (2022-04-28) Delta Lake Community Office Hours (2022-04-19) Delta Lake Community Office Hours (2022-04-19) Delta Lake Community Office Hours (2022-03-31) Delta Lake Community Office Hours (2022-03-03) Delta Lake Community Office Hours (02/17/2022) Delta Lake Community Office Hours (2022-02-03) Presto / Delta Lake Community Office Hours Delta Lake Community Office Hours

Delta Hack 2021

A collection of live streamed sessions from #DeltaHack 2021

Delta Hack: Introduction to Delta Lake Delta Hack: Create Reliable Tables with Delta Lake & Apache Spark in under 10min Delta Hack: Making a first pull request to delta-rs Delta Hack: Live demo of kafka-delta-ingest Delta Hack: Committers AMA and Happy Hour

delta-rs development meetings

2021-10-26 delta-rs open development meeting 2021-09-17 delta-rs open development meeting 2021-08-31 delta-rs open development meeting 2021-04-13 delta-rs open development meeting 2021-04-27 delta-rs open development meeting 2021-06-08 delta-rs open development meeting 2021-04-13 delta-rs open development meeting 2021-03-30 delta-rs open development meeting 2021-03-16 delta-rs open development meeting 2021-03-02 delta-rs open development meeting 2021-02-02 delta-rs open development meeting 2021-01-19 delta-rs open development meeting 2021-02-02 delta-rs open development meeting 2021-01-19 delta-rs open development meeting