cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 

pipeline config DAB

prakharsachan
New Contributor

yesterday

I am deploying DLT pipeline in dev environment using DABs. source code is in a python script file. In the pipeline's yml file the configuration key is set to true(with all correct indentations), yet the pipeline isnt deploying in the continuous mode. Whats the reason?

0 Kudos
1 REPLY 1

szymon_dybczak
Esteemed Contributor III

yesterday - last edited yesterday

Hi @prakharsachan ,

Continuous must be set inside the pipeline resource definition, not under configuration.

The configuration block in a SDP (former DLT) pipeline definition is for Spark/pipeline settings (key-value string pairs passed to the runtime), not for pipeline deployment properties. So if you have something like:

szymon_dybczak_1-1776108303813.png

 

Databricks will just ignore it as an unrecognized Spark config. The fix is to promote continuous to a top-level property of the pipeline resource:

szymon_dybczak_0-1776108283652.png

 

 

 

 

Announcements