’ve created a notebook that uses Spark Structured Streaming and runs continuously, so I’ve deployed the corresponding Databricks job using the continuous trigger mode.
What I’d like is for this job to start automatically only in certain environments (for example, prod) when deploying with DABs.
I noticed there is a lifecycle property with a started field that appears to control the startup state during deployment, but according to the documentation, this only applies to apps, clusters, and SQL warehouses — not jobs.
For scheduled batch jobs, we can configure the pause state (PAUSED or UNPAUSED) per environment, which provides a nice way to control whether jobs start automatically after deployment. I’m looking for similar functionality for continuous/always-on streaming jobs.
One option is to handle this in the CI/CD pipeline, for example:
if target = prod:
databricks bundle run streaming_job
But I’m wondering if there’s a more elegant or native way to manage the default start behavior of continuous jobs across environments when using DABs.
