Azure Asset Bundle deploy removes the continous: true configuration

zuzsad — Fri, 20 Dec 2024 14:20:07 GMT

I have this pipeline configuration that I'm deploying using Azure Asset Bundles:

ingest-pipeline.test.yml

```

resources:
pipelines:
ingest-pipeline-test:
name: ingest-pipeline-test-2
clusters:
- label: default
node_type_id: Standard_D4ads_v5
autoscale:
min_workers: 1
max_workers: 2
mode: ENHANCED
libraries:
- notebook:
path: ../notebooks/Multi-table pipeline.py
target: test_bundle_project_${bundle.target}
continuous: true
development: true
catalog: test_databricks_premium
configuration:
bundle.sourcePath: ${workspace.file_path}/notebooks
```
The notebook is creating a couple of streaming tables (DLT). The YAML validates and deploys just fine and I can see my job under "Delta Live Tables". However, the deployed pipeline has mode "Triggered" and not "Continous". If I check the YAML of the deployed pipeline (by clicking the pipeline name in the Databricks UI, then the kebab menu next to "Development" and selecting "View settings YAML", I see that "continous: true" is gone (other than that the configuration is basically the same as my YAML file above). Is this a bug or am I configuring the job incorrectly? I actually looked at the settings YAML for another continous DLT job that I already had running in Databricks, which was created using the UI, so that's where I got the "continous: true" from.

Re: Azure Asset Bundle deploy removes the continous: true configuration

Walter_C — Fri, 20 Dec 2024 15:02:50 GMT

Which CLI version are you using? Is it the latest version?

Re: Azure Asset Bundle deploy removes the continous: true configuration

zuzsad — Fri, 20 Dec 2024 20:56:17 GMT

I'm using Databricks CLI v0.237.0

Re: Azure Asset Bundle deploy removes the continous: true configuration

Walter_C — Tue, 24 Dec 2024 02:25:01 GMT

I just got confirmation internally that as of now Continuous is disabled in mode: development

topic Re: Azure Asset Bundle deploy removes the continous: true configuration in Data Engineering

Azure Asset Bundle deploy removes the continous: true configuration

Re: Azure Asset Bundle deploy removes the continous: true configuration

Re: Azure Asset Bundle deploy removes the continous: true configuration

Re: Azure Asset Bundle deploy removes the continous: true configuration