unzip twice the same file not executing

[RantoB]

Hi, 

I need to unzip some files that are ingested but when I unzip twice the same zipped file, the unzip command does not execute :

As suggesgted in the documentation I did :

import urllib 
urllib.request.urlretrieve("https://resources.lendingclub.com/LoanStats3a.csv.zip", "/tmp/LoanStats3a.csv.zip")

%sh
unzip /tmp/LoanStats3a.csv.zip

but when it apply again unzip, command never execute and seems to be blocked in a no out loop.

Thanks for you help.

[Prabakar]

Hi @Bertrand BURCKER​ as you have mentioned your zip file is large, can you let us know the size of the file?

Also, have you tried with a smaller zip file, and what is the result?

[RantoB]

My file is 180MiB. For information, the culster is a single node standard_F4s

[Hubert-Dudek · Accepted solution]

Another problem is that dbfs storage doesn't support random writes (used by zip):

Does not support random writes. For workloads that require random writes, perform the operations on local disk first and then copy the result to

/dbfs

source: https://docs.databricks.com/data/databricks-file-system.html#local-file-api-limitations