@RobertoBruno The tfstate used is actually the one stored in the Databricks Workspace, not on the local filesystem. So providing you keep using the same root_path for your DAB, it should still correctly clean-up any jobs you remove from your code in GIT.

e.g. root_path for staging should be fixed for each DAB, and only the Service Principle running the CI process should be able to run this process:

  staging:
    workspace:
      host: https://adb-123456789.1.azuredatabricks.net/
      root_path: /Workspace/DAB/${bundle.name}/${bundle.target}

 A way to demonstrate this is:

  1. Create new job in DAB locally
  2. Run bundle deploy
  3. Job appears in Databricks Workspace's list of jobs
  4. Delete new job locally
  5. Delete local .databricks folder (containing local tfstate). This simulates a new CI/CD run on a fresh build agent.
  6. Run bundle deploy
  7. Result: Job is deleted from the Databricks Workspace