โve created a notebook that uses Spark Structured Streaming and runs continuously, so Iโve deployed the corresponding Databricks job using the continuous trigger mode.
What Iโd like is for this job to start automatically only in certain environments (for example, prod) when deploying with DABs.
I noticed there is a lifecycle property with a started field that appears to control the startup state during deployment, but according to the documentation, this only applies to apps, clusters, and SQL warehouses โ not jobs.
For scheduled batch jobs, we can configure the pause state (PAUSED or UNPAUSED) per environment, which provides a nice way to control whether jobs start automatically after deployment. Iโm looking for similar functionality for continuous/always-on streaming jobs.
One option is to handle this in the CI/CD pipeline, for example:
if target = prod:
databricks bundle run streaming_job
But Iโm wondering if thereโs a more elegant or native way to manage the default start behavior of continuous jobs across environments when using DABs.
