Hi,
We are currently working on resolving the issue without exceeding the quota, if possible.
Here are our observations:
1. Notebook Execution Time Difference:
- When running a notebook activity using the interactive cluster, it completed in 5 minutes and 7 seconds. However, the same notebook took 1 hour and 32 minutes to execute using the job cluster.
Attaching the configuration details for both the clusters:
Interactive Cluster:
New Job Cluster:
2. Job Cluster having Node Type Standard_D8ds_v4 :
- We are getting an error ‘Node type Standard_D8ds_v4 is not supported’ when we tried using new job cluster having cluster node type as Standard_D8ds_v4, however it’s available in the list we are getting in the error message.
It would be helpful if you could check these issues and provide us with your suggestions.
