Databricks Community

Swathik

I am designing a metadata‑driven ETL framework to migrate approximately 500 tables from Db2 to PostgreSQL.After reviewing multiple design patterns and blog posts, I am uncertain about the recommended approach for storing ETL metadata such as source s...

Swathik

I am in the process of designing a Medallion architecture where the data sources include REST API calls, JSON files, SQL Server, and Azure Event Hubs.For the Silver and Gold layers, I plan to leverage Delta Live Tables (DLT). However, I am seeking gu...

Swathik

Thank you. This helps a lot

Swathik

Ok but is it still suggested to have a for each loop even for 500 tables?? Also if one of the table fails in the loop I think it would be difficult to go over all the 500 tables.

Swathik · 05-21-2025

I have completed the courses but not received any voucher yet. My email id Swathi.kommana@gmail.com. could you please share me the voucher @Jim_Anderson

Swathik · 04-09-2025

Im getting this error while trying to access Learning path under Associate Data Engineer but able to access Advanced Data Engineer course

Databricks Community

User Stats

User Activity

Best practices for the meta data driven ETL framework

Best Practices for implementing DLT, Autoloader in Workflows

Re: Best practices for the meta data driven ETL framework

Re: Best practices for the meta data driven ETL framework

Re: Virtual Learning Festival: 9 April - 30 April

Re: Virtual Learning Festival: 9 April - 30 April