cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Any recommendations on instance type for z-order / vacuum/ optimize ?

Anonymous
Not applicable
 
1 ACCEPTED SOLUTION

Accepted Solutions

Anand_Ladda
Honored Contributor II

For Delta in general having Delta cache accelerates data reads by creating copies of remote files in nodesโ€™ local storage using a fast intermediate data format. The data is cached automatically whenever a file has to be fetched from a remote location. Successive reads of the same data are then performed locally, which results in significantly improved reading speed. And this can be useful even when performing merges where data has to be read.

https://docs.databricks.com/delta/optimizations/delta-cache.html#use-delta-caching

The recommended (and easiest) way to use Delta caching is to choose a Delta Cache Acceleratedโ€”

i3 or i3en seriesโ€”worker type when you configure your cluster. Such workers are enabled and configured for Delta caching.

View solution in original post

1 REPLY 1

Anand_Ladda
Honored Contributor II

For Delta in general having Delta cache accelerates data reads by creating copies of remote files in nodesโ€™ local storage using a fast intermediate data format. The data is cached automatically whenever a file has to be fetched from a remote location. Successive reads of the same data are then performed locally, which results in significantly improved reading speed. And this can be useful even when performing merges where data has to be read.

https://docs.databricks.com/delta/optimizations/delta-cache.html#use-delta-caching

The recommended (and easiest) way to use Delta caching is to choose a Delta Cache Acceleratedโ€”

i3 or i3en seriesโ€”worker type when you configure your cluster. Such workers are enabled and configured for Delta caching.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.