Databricks

Luu · 08-07-2023

Hi all,recently I am facing a strange behaviour after an OPTIMZE ZOrder command. For a large table around (400 mio. rows) I executed the OPTIMIZE command with ZOrder for 3 columns. However, it seems that the command does not have any effect and the c...

Luu · 08-09-2023

I have test it with setting the config: spark.conf.set("spark.databricks.delta.optimize.preserveInsertionOrder", false). Unfortunately, the OPTIMIZE ZORDER BY still does not process anything....totalConsideredFiles: 2137totalFilesSkipped: 2137...

Luu · 08-08-2023

@Tharun-Kumar thank you for the information. I was not aware of this configuration parameter.I will check this.Thank you

Luu · 08-07-2023

Thank you for the quick reply Kaniz. I can provide more information to your questions.The table which I am considering is a Delta table with many columns (194 columns)The table is partitioned by two columnsThe table is large (around 400 mio rows, 213...

Databricks

User Stats

User Activity

OPTIMZE ZOrder does not have an effect

Re: OPTIMZE ZOrder does not have an effect

Re: OPTIMZE ZOrder does not have an effect

Re: OPTIMZE ZOrder does not have an effect