Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-06-2025 09:18 AM
@RobertoBruno The tfstate used is actually the one stored in the Databricks Workspace, not on the local filesystem. So providing you keep using the same root_path for your DAB, it should still correctly clean-up any jobs you remove from your code in GIT.
e.g. root_path for staging should be fixed for each DAB, and only the Service Principle running the CI process should be able to run this process:
staging:
workspace:
host: https://adb-123456789.1.azuredatabricks.net/
root_path: /Workspace/DAB/${bundle.name}/${bundle.target}
A way to demonstrate this is:
- Create new job in DAB locally
- Run bundle deploy
- Job appears in Databricks Workspace's list of jobs
- Delete new job locally
- Delete local .databricks folder (containing local tfstate). This simulates a new CI/CD run on a fresh build agent.
- Run bundle deploy
- Result: Job is deleted from the Databricks Workspace