jcozar
Contributor
In response to Deekay

‎06-18-2025 07:17 AM

Hi @Deekay ,

in the end what I did:

  • Send from postgres databases to azure eventhub using debezium server
  • Use spark streaming jobs to save from eventhub to azure storage account as raw data. In this way, I have full control to configure how data is written (for example, for small master data, it is directly consolidated using delta updates)
  • Use dlt or spark streaming jobs to create bronze data from raw data, writing tables to unity catalog
  • Use dlt or spark streaming jobs to create the rest of the ETLs, from bronze to silver and gold
I hope that this helps you! I also would like to hear from your opinion 🙂