cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Bundle Validation Error: Volume lifecycle Field Rejected Despite Being in Schema

animeshjain
New Contributor II

Environment:

  • Bundle engine: direct
  • Resource type: Volume (Unity Catalog)

Problem:

I'm trying to use the lifecycle field with prevent_destroy: true on a Unity Catalog volume resource in my Declarative Automation Bundle (DAB). the validation fails with:

Error: Error while loading volume
at resources.volumes.raw_zone

ValueError: Unexpected field 'lifecycle' for class Volume

```yaml

resources:
volumes:
raw_zone:
catalog_name: "datazone_dev"
schema_name: "source"
name: "raw_zone"
comment: "Volume created for raw zone data storage"
volume_type: "MANAGED"
lifecycle:
prevent_destroy: true
grants:
- principal: "user-name1"
privileges:
- READ_VOLUME
- WRITE_VOLUME

```

 
When I run databricks bundle schema, the JSON schema clearly shows that lifecycle is a valid field for Volume resources:
"resources.Volume": {
"type": "object",
"properties": {
"catalog_name": { ... },
"comment": { ... },
"grants": { ... },
"lifecycle": {
"description": "Settings that control the deployment lifecycle of the resource, such as preventing it from being destroyed.",
"$ref": "#/$defs/github.com/databricks/cli/bundle/config/resources.Lifecycle"
},
"name": { ... },
"schema_name": { ... },
"volume_type": { ... }
}
}

is there something where I am wrong to get this error 

animeshjain_0-1783340968236.pnganimeshjain_1-1783340987403.png

 

4 REPLIES 4

balajij8
Contributor III

What you are seeing is expected behavior when the local execution environment is out of sync with the latest target schema definitions. prevent_destroy: true is a fully supported property for Unity Catalog Volume resources within Databricks Asset Bundles (DABs).  The ValueError: Unexpected field 'lifecycle' for class Volume error is a strict validation failure thrown by an older Databricks CLI bundle engine that does not yet recognize the lifecycle class for Volume resource types.

You can follow below

  • If you are running Databricks CLI version older than v1.6.0 - The local environment you use lacks the updated Python-based schema definitions that introduced volume lifecycle management. You must upgrade your Databricks CLI to v1.6.0 or higher. You can verify your current version by executing databricks --version in the terminal.

  • If your CLI is already up to date (v1.6.0+) but validation still fails - It can be due to corrupted local state cache or a YAML syntax misalignment. Delete the hidden .databricks caching directory in the project root to clear the state, ensure strict YAML indentation under the raw_zone block, and execute databricks bundle validate to force a clean parse.

  • If you are experiencing this error within an automated CI/CD pipeline - The pipeline runner (GitHub Actions, Azure DevOps, GitLab) is pulling a stale, cached version of the Databricks CLI. Ensure the pipeline configuration explicitly specifies the latest CLI version tag in its setup step.

Hi thanks for reply 
I am running cli version 1.6.0 and still getting this error ,i also removed the .databricks directory from the project root
attaching screenshot for the same 

animeshjain_0-1783679184456.png

 

@animeshjain I think the error is coming from the local Python bundle parser, not from the schema itself, your CLI is showing a schema where lifecycle is valid for volume, but may be the installed databricks.bundles.volumes runtime that actually parses the YAML is older and still rejects that field.

You can try checking if multiple Databricks CLI installs are present by running below

which databricks
databricks --version
python -c "import databricks.bundles.volumes as v; print(v.__file__)"

and if that points to an old virtualenv/site-packages path, do a clean reinstall of the Databricks CLI and upgrade to the latest available version rather than staying on 1.6.0. Removing .databricks alone might not fix this because the failure is in the installed parser code, not bundle state.

balajij8
Contributor III

Hi, 

The configuration is valid and the CLI version supports the lifecycle field. However, the error is most likely coming from outdated Databricks Python packages in your virtual environment at /home/animesh/datazone/.venv/. CLI uses the packages from the venv to parse and validate the bundle. Those packages are from an older version that doesn't recognize lifecycle as a valid field for volumes and hence the Unexpected field 'lifecycle' for class Volume error.

You can activate your virtual environment and upgrade the databricks-sdk & all other packages in the environment. You can upgrade to python 3.13.2 if feasible for better management. lifecycle.prevent_destroy field will be recognized and your bundle will validate and deploy successfully after its upgraded.