Databricks Community

Master_DataBric

Is it possible to define an expectation in DLT pipeline using multiple columns?For example, my source has two fields - Division, Material_Number. For division 20, material number starts with 5; for 30 material number starts with 9.Can we have this ...

Master_DataBric

You can extract the files and save them on cloud storage(s3/ADLS/GCS).Then you can use the appropriate cloud CLI to read the files from cloud storage. I think this is more scalable and can be achieved programmatically(i.e via your batch job).

Databricks Community

User Stats

User Activity

Expectation in DLT using multiple columns

Re: Need to extract data from delta tables and need to move it to on-prem, what's the best approach