Yaadhu
New Contributor II

you can create the pool instance in the databricks under compute/pool and assign the value in the json of the DLT pipeline. With this, we will control on pool min workers and max workers and the reuse of the pools available by other pipelines. 

"node_type_id": "i4i.xlarge",
"driver_node_type_id": "i4i.2xlarge",
"instance_pool_id": "pool id from the pool configured in pools ",
"driver_instance_pool_id": "pool id from the pool configured in pools ",
"autoscale": {
"min_workers": 1,
"max_workers": 6,
"mode": "ENHANCED"
}