cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Difference between Delta Live Tables and Multitask Jobs

Anonymous
Not applicable

When should I use one over the other? There seems to be an overlap of some functionality

1 ACCEPTED SOLUTION

Accepted Solutions

aladda
Databricks Employee
Databricks Employee
Delta Live Tables are targeted towards building an ETL pipeline where several Delta tables are interconnected from a flow perspective and in a single notebook. Multi-task Jobs is more generic orchestration framework that allows you to execute various Notebooks in a flow

View solution in original post

2 REPLIES 2

User16783855117
Contributor II
Delta Live Tables provides and API for 'declarative data flow'. Instead of writing the code to manually read in data, apply transformations, and write it out, Delta Live tables instead asks you to declare the input source, the transformations, and the intended output source. The managed runtime will automatically optimize under the covers and perform the required reads, bookmarking, and compute logistics. It is very similar to how Terraform is declarative for building cloud infrastructure, whereas Delta Live Tables are declarative for data tables.

aladda
Databricks Employee
Databricks Employee
Delta Live Tables are targeted towards building an ETL pipeline where several Delta tables are interconnected from a flow perspective and in a single notebook. Multi-task Jobs is more generic orchestration framework that allows you to execute various Notebooks in a flow