Databricks Auto-Loader vs. Delta Live Tables
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-15-2021 05:59 AM
What is the difference between Databricks Auto-Loader and Delta Live Tables? Both seem to manage ETL for you but I'm confused on where to use one vs. the other.
- Labels:
-
Delta
-
Delta Live Tables
-
DLT
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-15-2021 08:13 AM
Auto-Loader allows incrementally data ingestion into Delta Lake from a variety of data sources while Delta Live Tables are used for defining end-to-end data pipelines by specifying the data source, the transformation logic, and destination state of the data — instead of manually stitching together siloed data processing jobs.
In effect AutoLoader would be a piece of the DLT pipeline specifically on the data source side as raw data is incrementally brought into Delta Lake. DLT would then pick up from there for the downstream transformation and processing of the data as illustrated here
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-26-2024 02:27 PM
You say "...__would__ be a piece..." and "...DLT __would__ pick up...".
Is DLT built upon AL?

