cancel
Showing results for 
Search instead for 
Did you mean: 
Warehousing & Analytics
cancel
Showing results for 
Search instead for 
Did you mean: 

how to create an Indexes for Databricks tables

User16826994223
Honored Contributor III

can I know How can I create and use index in databricks

1 REPLY 1

User16826994223
Honored Contributor III

partitioning could be seen as kind of index - it helps you to directly query the folder in partition which has relevant data leaving all other partition folder untouched.

there is another feature in data bricks Delta - Data Skipping. When writing data to Delta, the writer is collecting statistics (for example, min & max values) for first N columns (32 by default) and write that statistics into Delta log, so when we filter data by indexed column, we know if given file may contain given data or not. Another indexing technique for databricks delta is bloom filtering that is shows if the specific value is definitely not in the file, or could be in the file.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.