Databricks Community

Dharinip

1. How to create a Delta Table with PK and FK constraints for a streaming source data?2. When the streaming data in the silver layer gets updated, will the delta table also be updated?My use case is:We have a streaming data in the silver layer as SCD...

Dharinip

We have the following use case:We receive raw form of data from an application and that is ingested in the Iron Layer. The raw data is in the JSON FormatThe Bronze layer will the first level of transformation. The flattening of the JSON file happens ...

Dharinip · 05-24-2023

I am getting this type of erro "com.databricks.spark.sqldw.SqlDWConnectorException: Exception encountered in Azure Synapse Analytics connector code" while loading the data into Synapse. I am able to load other tables with the same connection. Not sur...

Dharinip

Hi,To your question, the dataset is large and will be growing as well. In my use case the Silver layer contains a USER table(streaming Delta live table) like below:USER table:batchidIDNamestartdateEnddateakjfdhjsa123ABC11/01/202411/02/2024adjagfua123...

Dharinip

Hi Madams,Thank you for your reply. Let me explain in detail about the use case that we have.The raw data is ingested in the Iron layer on batch processing. The file would be in JSON format that we receive in batches. IRON Layer - The first level whe...

Databricks Community

User Stats

User Activity

Create a Delta Table with PK and FK constraints for a streaming source data

How to decide on creating views vs Tables in Gold layer?

Connection Error

Re: How to decide on creating views vs Tables in Gold layer?

Re: How to decide on creating views vs Tables in Gold layer?