Databricks YAML files

[prakharsachan]

when i change the yam file(not the resources one but can be any config yml) locally and then redeploy that bundle , the changes are not visible in the remote yml but other files(.py, .sql) do reflect these changes. FYI: during dev , i make changes in the remote version directly in both config yml and other files too. Is this a databricks feature or a technical issue?

[iyashk-DB]

YAML files aren't intentionally excluded from bundle sync, so this is almost certainly a config or snapshot issue rather than a product decision.

A few things to check:

1. .gitignore is the most common cause. Bundle sync respects your .gitignore by default. If your repo has *.yml, *.yaml, or specific config file patterns listed there, those files will be silently skipped during bundle deploy even though .py and .sql files sync fine. Check your .gitignore (and any nested .gitignore files) for yaml-related patterns.

2. Direct remote edits corrupt the sync snapshot. You mentioned you're editing config YAML directly in the workspace during dev. The sync uses a local snapshot to track what was last uploaded. When you edit a file remotely, that snapshot doesn't update. Next time you run bundle deploy, the sync compares your local file against the snapshot. If the local file looks unchanged from the snapshot's perspective, it skips the upload entirely and leaves the remote as-is. This can make it look like local YAML changes aren't making it through.

To force a full re-sync and break out of this state, run:

databricks bundle deploy --full

This bypasses the snapshot and re-uploads every file, so your local version wins regardless of what's on the workspace.

3. Going forward, treat bundle deploy as the source of truth for your workspace files and avoid editing remote copies directly. If you need to iterate quickly on config, edit locally and either re-deploy or use databricks bundle sync --watch to continuously push changes as you save.

[prakharsachan]

About your point 2 , the situation is only in the case of yaml files , but not with other file formats such as .py or .sql. Even when other format files are changed in remote directly, these files are updated and synced with local except the .yml file (only when its modified remotely) after redeployement. 

[AmitDECopilot]

I think this is mostly expected behavior with Databricks Asset Bundles, not really a bug.

In bundle deployment, the local bundle config is usually the source of truth. So when you change databricks.yml or any included YAML locally and run deploy, Databricks uses that YAML to update the actual resources like jobs, pipelines, variables, permissions, etc. But the YAML file itself may not always show up in the remote workspace the same way .py or .sql files do.

Few things I would check:

1. Is that YAML file included in the main databricks.yml using the include section?
2. Run databricks bundle validate -t <target> and see if your local change is actually getting picked up.
3. Deploy from the bundle root using databricks bundle deploy -t <target>.
4. Avoid changing the remote YAML directly during development, because then it becomes confusing which version is the real source  local or remote.

My usual approach would be: keep YAML changes local/Git-based, validate, deploy, and then verify the actual deployed job settings rather than only checking whether the remote YAML file changed visually.

So in your case, I would first confirm whether the YAML is part of bundle config or just a synced file expectation

[binlogreader]

@prakharsachan 

What you're seeing is designed behavior, not a bug.

When you run `bundle deploy`, the CLI does not upload everything in your bundle folder. It builds the upload list by walking the directory and skipping anything your .gitignore matches, then adding whatever you list under `sync.include`, then removing `sync.exclude`.

Your .py and .sql files are normal tracked code, so they upload every time. Config yml files are exactly the kind of thing that ends up in .gitignore (environment values, generated settings), and a gitignored file is silently left out. No warning, no error, the deploy still succeeds.

So quite likely, your config yml is gitignored and local edits never upload. The remote yml you're looking at is the copy you've been editing manually in the workspace, and the deploy never touches it.

You can use two commands for validations:

```
git check-ignore -v path/to/your/config.yml
```

If this prints a rule, that's the issue. It shows which line of which .gitignore file excludes your yml.

```
databricks bundle sync --output json
```

This prints every file the deploy actually uploads. If your yml never appears in the list even right after a local edit, that proves the file isn't part of the sync at all, rather than anything cache related.

The fix is to whitelist the file in databricks.yml, which overrides the gitignore skip:

```yaml
sync:
include:
- configs/*.yml
```