Sidhant07
Databricks Employee
Databricks Employee

Hi,

Possible Causes and Solutions

  1. Network Configuration:
    • The private connectivity setup might be affecting DLT clusters differently.
  2. Cluster Configuration:
    • Ensure DLT clusters are properly sized for the workload.
    • Consider using a larger driver node for complex transformations.
  3. Dependency Management:.
    • Consider using cluster pools to reduce startup times