Difference between libraries dlt and dp

[yit]

In all Databricks documentation, the examples use import dlt to create streaming tables and views. But, when generating sample Python code in ETL pipeline, the import in the sample is:

import pyspark import pipelines as dp

Which one is the correct library? Are there significant differences between these two?

I could not find any comparison between these two.

[szymon_dybczak · Accepted solution]

Hi @yit ,

Basically, they are the same thing. Not so long ago,  they renamed DLT (Delta Live Tables) to Lakeflow Declartive Pipelines. So I think from now on in all materials you will see new name for this library: pipelines (instead of dlt)

 guess they didn't managed to change old name in all their docs yet, so you can see here and there old name for that library.

[nayan_wylde]

@yit Functionally, they are equivalent concepts (declarative definitions for streaming tables, materialized views, expectations, CDC, etc.). The differences you’ll notice are mostly naming/ergonomics:

Module name:
Databricks docs & most existing notebooks: import dlt

https://docs.databricks.com/aws/en/dlt/python-dev

Spark guide & some generated samples: from pyspark import pipelines as dp

https://spark.apache.org/docs/_site/declarative-pipelines-programming-guide.html