Does anyone know why the optimize does not complete?

irfanaziz
Contributor II

I feel there is some issue with a few partitions of the delta file. The optimize runs fine and completes within few minutes for other partitions but for this particular partition the optimize keeps running forever.

OPTIMIZE delta.`/mnt/prod-abc/Inis/data/comm.delta/` where date='2020-03-05'

Is there a way i could check what is going on or any way to check if there are any issues with the files in this partition or in the deltalog?

I can query the table and i can read this partition but i cannot write this particular partition back to another path(the write keeps running forever). I am running optimize as merge command has been timing out on this delta file. I feel it was a 'z order by(id)' which screwed the distribution. Now i am trying to optimize without the z order.

irfanaziz
Contributor II

We had to remove the partition and insert the data again to fix the issue.

Anonymous
Not applicable

@nafri A​ - Thank you for letting us know.