Databricks Community

issa · 11-29-2024

I have a job in Workflows thatt runs two DLT pipelines, one for Bronze_Transaction and on for Silver_Transaction. The reason for two DLT pipelines is because i want the tables to be created in bronze catalog and erp schema, and silver catalog and erp...

issa · 12-19-2024

Final solution for the Bronze:# Define view as the source@dlt.viewdef Transactions_Bronze_View(): return ( spark.readStream.format("cloudFiles") .option("cloudFiles.format", "json") .option("inferSchema", True) .option(...

issa · 12-19-2024

@filipniziol - its resolved now, thanks. What I did to solve it was to delete the tables and runt everything again. Even though I did a full refresh from DLT, that didnt seem to do the trick. So I deleted the tables thinking that it could be some cac...

issa · 12-18-2024

Yes, this part is fine:import dltfrom pyspark.sql.functions import col, lit, current_timestampfrom pyspark.sql import functions as Fstorage_account = 'xxxxx'container = 'xxxxxx'landing_path = 'xxxxxx/Transactions'landing_json_path = f"abfss://{contai...

issa · 12-09-2024

Hi @filipniziol and @ozaaditya,Thank you both for you input. I changed the code, since I figured that the SCD should be on the bronze layer and that i then should filter out open rows in silver.However, that dosent work.My idea was:Bronze layer:impor...

Databricks Community

User Stats

User Activity

How to access bronze dlt in silver dlt

Re: How to access bronze dlt in silver dlt

Re: How to access bronze dlt in silver dlt

Re: How to access bronze dlt in silver dlt

Re: How to access bronze dlt in silver dlt