Network time-out

[GJL-Geophysics]

When trying to download the public domain seismic data of Equinor’s Volve field (https://www.equinor.com/energy/volve-data-sharing) using a browser (Edge or Chrome) this fails after 2 hours due to network errors. Resuming only provides once off relieve until it fails again. Laptop, Lenivo running Win11, settings are such to keep system active (never sleep). Network speed is 600+ Mbps download and 100+ Mbps upload. Browser downloads only at 8-9 Mbps. With 2.1 and 1.2 Tb for the two files, download time is initially estimated between 2-5 days!

Any suggestions how to resolve?

Is there an FTP option with faster downloads (perhaps not for personal free databricks account)?

[SumeshKashyap]

The browser is the bottleneck, not your line: it pulls a 2 TB file over a single HTTP connection, so you get ~8 Mbps and any blip kills the transfer. The Volve data sits in Azure Blob Storage, so download it with AzCopy instead. AzCopy splits the file into many parallel chunks (it should use most of your 600 Mbps) and can resume a failed job where it stopped. There's no FTP option.

1. Get the Blob SAS URL from the Volve data-sharing page after accepting the licence (it's the same link Azure Storage Explorer uses).

2. Install AzCopy (https://learn.microsoft.com/azure/storage/common/storage-use-azcopy-v10) and run:

azcopy copy "<SAS URL to the file or folder>" "D:\volve" --recursive

3. If it drops, run azcopy jobs list, then azcopy jobs resume <job-id> to continue from the last completed chunk instead of starting over.

If you plan to process the data in Databricks anyway, skip the laptop: point the same azcopy copy at your own storage account (a SAS URL for the destination container). That's a server-to-server copy inside Azure and takes minutes to hours instead of days. Make sure the target disk has ~3.5 TB free if you do go local.