Does anyone know why the optimize does not complete?

[irfanaziz]

I feel there is some issue with a few partitions of the delta file. The optimize runs fine and completes within few minutes for other partitions but for this particular partition the optimize keeps running forever.

OPTIMIZE delta.`/mnt/prod-abc/Inis/data/comm.delta/` where date='2020-03-05'

Is there a way i could check what is going on or any way to check if there are any issues with the files in this partition or in the deltalog?

I can query the table and i can read this partition but i cannot write this particular partition back to another path(the write keeps running forever). I am running optimize as merge command has been timing out on this delta file. I feel it was a 'z order by(id)' which screwed the distribution. Now i am trying to optimize without the z order.

[Kaniz]

Hi @ irfanaziz! My name is Kaniz, and I'm a technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers on the Forum have an answer to your questions first. Or else I will follow up shortly with a response.

[irfanaziz]

We had to remove the partition and insert the data again to fix the issue.

[Anonymous]

@nafri A​ - Thank you for letting us know.