Databricks Community
Data Engineering
Re: How to optimize jobs performance
Lakshay
Databricks Employee
In response to
RajeshRK
Options
Mark as New
Bookmark
Subscribe
Mute
Subscribe to RSS Feed
Permalink
Print
Report Inappropriate Content
03-17-2023
12:06 PM
Sure. You can also try the below suggestions:
Use compute-optimized Node Type. Currently, you are using a Memory-optimized one.
Run the job with spark.sql.shuffle.partitions auto
