AbhaySingh
Databricks Employee
Databricks Employee

Here are my recommendations:

1. Databricks Asset Bundles (DABs) for reusable workflows

  2. API-based triggering and Run Job Tasks for cross-repo workflows

  3. Instance Pools as the #1 game-changer for cluster optimization (5-10 seconds vs 5-10 minutes startup)