Debayan
Databricks Employee
Databricks Employee

Hi, the workaround seems to be is to use:

instance_pool_id directly in your pipeline settings.

"clusters": [

{

"label": "default",

"instance_pool_id": "0713-080933-line63-pool-r1jmrzte",

"num_workers": 5

}

],