cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Databricks Auto-Loader vs. Delta Live Tables

User16826992185
Databricks Employee
Databricks Employee

What is the difference between Databricks Auto-Loader and Delta Live Tables? Both seem to manage ETL for you but I'm confused on where to use one vs. the other.

2 REPLIES 2

aladda
Databricks Employee
Databricks Employee

Auto-Loader allows incrementally data ingestion into Delta Lake from a variety of data sources while Delta Live Tables are used for defining end-to-end data pipelines by specifying the data source, the transformation logic, and destination state of the data โ€” instead of manually stitching together siloed data processing jobs.

In effect AutoLoader would be a piece of the DLT pipeline specifically on the data source side as raw data is incrementally brought into Delta Lake. DLT would then pick up from there for the downstream transformation and processing of the data as illustrated here

Steve_Lyle_BPCS
New Contributor II

You say "...__would__ be a piece..." and "...DLT __would__ pick up...".
Is DLT built upon AL?

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group