topic Re: Uploading CSV to Databricks community edition in Data Engineering

Uploading CSV to Databricks community edition

Khaled — Wed, 08 Dec 2021 19:58:42 GMT

When I upload a csv file of size 1 GB from my PC the in the upload place, it is uploading untill the file reach some point and disappear for example it reach 600 MB and disappear from that place

Re: Uploading CSV to Databricks community edition

Prabakar — Wed, 08 Dec 2021 20:35:45 GMT

Hi @Khaled ALZHARANI , as the community edition is free for learning there are certain limitations. To understand why this is failing, you can get the HAR traces while uploading the file. This will show you the progress and if it fails you might get the status in the HAR.

Re: Uploading CSV to Databricks community edition

Khaled — Wed, 08 Dec 2021 20:39:41 GMT

Where can I find HAR and is it a shortcut for what ?

Re: Uploading CSV to Databricks community edition

Prabakar — Thu, 09 Dec 2021 08:41:19 GMT

This is the procedure for Google Chrome. For other browsers, see G Suite Toolbox HAR Analyzer.

Open Google Chrome and go to the page where the issue occurs.
In the Chrome menu bar, select View > Developer > Developer Tools.
In the panel at the bottom of your screen, select the Network tab.
Look for a round Record button in the upper left corner of the Network tab, and make sure it is red. If it is grey, click it once to start recording.
Check the box next to Preserve log.
Click Clear to clear out any existing logs from the Network tab.
Reproduce the issue while the network requests are being recorded.
After you reproduce and record the issue, right-click anywhere on the grid of network requests to open a context menu, select Save all as HAR with Content, and save the file to your computer.
Analyze the file using the HAR Analyzer tool.

Re: Uploading CSV to Databricks community edition

jose_gonzalez — Fri, 10 Dec 2021 23:17:20 GMT

Hi @Khaled ALZHARANI ,

I would also recommend to split up your CSV files into smaller files.