DLT pipelines in the same job sharing compute

[bradleyjamrozik]

If I have a job like this that orchestrates N DLT pipelines, what setting do I need to adjust so that they use the same compute resources between steps rather than spinning up and shutting down for each individual pipeline?

[bradleyjamrozik]

Edit: I'm using databricks asset bundles to deploy both the pipelines and jobs.

[shan_chandra]

@bradleyjamrozik  - under the DLT settings, notebooks can be listed all together. It will deploy a single compute resource for all the tasks. 

[bradleyjamrozik]

The different pipelines point to different catalogs/schemas so they have to be separated out.

[shan_chandra]

@bradleyjamrozik - I checked internally too. DLT pipelines cannot share resources.