- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-08-2023 08:48 AM
There are several potential reasons why your OPTIMIZE ZORDER command may not have had any effect on your table:
- The existing data files may already be optimally sorted based on the ZOrder and/or column ordering.
If the data is already optimized based on the ZOrder and column ordering, the OPTIMIZE ZORDER command will not have any effect. You can check the FILES section of the table details to see if the number of files or file size has changed after the optimization.
- The ZOrder columns may not have sufficient cardinality.
For the ZOrder optimization to be effective, the columns used for ZOrder should have high cardinality, meaning that the number of possible distinct values for the columns should be large. If the cardinality is low, then the data may not be distributed optimally across the files after the optimization.
- The OPTIMIZE ZORDER command may have failed.
To check if the optimize command has failed, you can look for any error messages in the cluster logs or use the DESCRIBE HISTORY command to view the history of the table and see if there were any failures or issues with the optimization.
Make sure that you are using Databricks Runtime version 6.4 or later, as the OPTIMIZE ZORDER command was introduced in that version.
You can also try running the command again and checking the logs for any error messages or contacting Databricks support for further assistance.
I hope that helps! Let me know if you have any other questions.