Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-09-2025 02:22 AM
This is one option yes, but ideally looking for to say
- Define cluster types once, usable by N packages that use databricks asset bundles
- Allow the bundle to just simply say run JOB.TASK step as Cluster_Type_1 for Production
Primary reasons for this is say we have common tagging strategies to apply on our clusters irrespective of asset bundle being deployed, and also common cluster configurations / spark conf setups.
A nice simple "middle ground" would be a common clusters.yml that can drop into any bundle if we decided to change the cluster_type_1 configuration and needed to replace it in 15 repos it would be nice and easy to change that one file.