aleksandra_ch
Databricks Employee
Databricks Employee

Hi @harshgrewal27 ,

Did you compare query profiles of a run in Env 3 vs Env 4? Check for:

  • Amount of data being processed per run
  • Longest running stages

This might also explain the execution time difference between Env 3 and Env 4.

Best regards,