MuthuLakshmi
Databricks Employee
Databricks Employee

@AlvaroCM 
The issue could be because the AWS doesn't have the instance type "waiting for resources" 

Can you use the instance type of the cluster which you were able to create using the clusters UI explicitly over the DLT pipeline as mentioned below.

"node_type_id": "m4.4xlarge",
"driver_node_type_id": "m4.4xlarge"