Databricks Community

I am using dlt through python to build a DLT pipeline. One of things I would like to do is to check that each incoming row does not exist in the target table; i want to be sure that each row is unique.I am confused because it seems like this is not p...

I am migrating a massive codebase to Pyspark on Azure Databricks,using DLT Pipelines. It is very important that code will be modular, that is I am looking to make use of UDFs for the timebeing that use modules and classes.I am receiving the following...

I need to insert pipeline_id and update_id in my Delta Live Table (DLT), the point being to know which pipeline created which row. How can I obtain this information?I know you can get job_id and run_id from widgets but I don't know if these are the s...