Deployed DABs job via Gitlab CICD. It is creating duplicate jobs.

ahen — Tue, 27 Aug 2024 23:02:14 GMT

We had error in DABs deploy and then subsequent retries resulted in a locked state
And as suggested in the logs, we use --force-lock option and the deploy succeeded
However, it created duplicate jobs for all assets in the bundle instead of updating the ones that are changed

Re: Deployed DABs job via Gitlab CICD. It is creating duplicate jobs.

Satyadeepak — Wed, 29 Jan 2025 00:44:46 GMT

@ahen When you used the --force-lock option during the Databricks Asset Bundle (DAB) deployment, it likely bypassed certain checks that would normally prevent duplicate resource creation. This option is used to force a deployment even when a lock is in place, which can lead to unintended consequences like duplicate jobs if the deployment logic does not correctly handle existing resources.

Use the bundle validate command to check the configuration and understand what changes will occur during deployment.

topic Deployed DABs job via Gitlab CICD. It is creating duplicate jobs. in Data Engineering

Deployed DABs job via Gitlab CICD. It is creating duplicate jobs.

Re: Deployed DABs job via Gitlab CICD. It is creating duplicate jobs.