cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Do I still need to use skew join hints if I have Adaptive Query Execution enabled?

User16826992666
Valued Contributor

From what I have read about AQE it seems to do a lot of what skew join hints did automatically. So should I still be using skew hints in my queries? Is there harm in using them?

1 ACCEPTED SOLUTION

Accepted Solutions

sajith_appukutt
Honored Contributor II

With AQE Databricks  has the most up-to-date accurate statistics at the end of a query stage and can opt for a better physical strategy and or do optimizations that used to require hints,

In the case of skew join hints, is recommended to rely on AQE skew join handling rather than use hints, because AQE skew join is automatic and in general performs better than the hint counterpart.

View solution in original post

1 REPLY 1

sajith_appukutt
Honored Contributor II

With AQE Databricks  has the most up-to-date accurate statistics at the end of a query stage and can opt for a better physical strategy and or do optimizations that used to require hints,

In the case of skew join hints, is recommended to rely on AQE skew join handling rather than use hints, because AQE skew join is automatic and in general performs better than the hint counterpart.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group