Code reusability for silver table transformations
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-30-2024 05:33 AM - edited 04-30-2024 09:37 AM
How/how many databricks notebooks should be created to populate multiple silver delta tables, all having different and complex transformations ? What's the best practice -
1. create a notebook each for a silver table ?
2. push SQL transformation logic to a config table ? And create a single reusble notebook for to be used by all the tables ? Or some other approach ?
PS - Created just a single notebook to convert the raw parquet tables to delta format, as the tranfromations were generic for all the bronze tables.