topic Re: Exporting csv files from Databricks in Data Engineering

Exporting csv files from Databricks

Sarvagna_Mahaka — Tue, 02 Nov 2021 13:30:24 GMT

I'm trying to export a csv file from my Databricks workspace to my laptop.

I have followed the below steps.

1.Installed databricks CLI

2. Generated Token in Azure Databricks

3. databricks configure --token

5. Token:xxxxxxxxxxxxxxxxxxxxxxxxxx

6. databricks fs cp -r dbfs:/your_folder destination/your_folder

I get the below error. Can anyone help?

Error: ConnectionError: HTTPSConnectionPool(host='%3cdatabricks-instance%3e', port=443): Max retries exceeded with url: /api/2.0/workspace/get-status?path=%2FFileStore%2Fshared_uploads%2Fwwid%40 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x000001E8649DDC08>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

Re: Exporting csv files from Databricks

Hubert-Dudek — Tue, 02 Nov 2021 13:59:29 GMT

Please check %USERPROFILE%\.databrickscfg file on Windows it should include

[DEFAULT]
host = <workspace-URL>
token = <personal-access-token>

Please validate host variable. Here are additional info how to get it https://docs.databricks.com/workspace/workspace-details.html#workspace-url

Please validate cli connection using some simple command for example

databricks workspace list

Re: Exporting csv files from Databricks

Anonymous — Tue, 02 Nov 2021 15:49:52 GMT

Depending on the file size, you can use display() in a notebook to download a CSV to your local laptop. It's possible for an admin has disabled this feature so you may not see it.

Re: Exporting csv files from Databricks

Sarvagna_Mahaka — Tue, 09 Nov 2021 10:15:13 GMT

Thanks Joseph, but the file I want to export is of a bigger size. It goes beyond the display limit of the rows.

Re: Exporting csv files from Databricks

Sarvagna_Mahaka — Tue, 09 Nov 2021 12:02:35 GMT

Thanks for the help, Hubert.

My host URL was not set correctly. I made the changes as suggested & that fixed the issue.

Re: Exporting csv files from Databricks

Hubert-Dudek — Wed, 10 Nov 2021 13:00:27 GMT

I am glad that it helped.

Re: Exporting csv files from Databricks

User16871418122 — Wed, 17 Nov 2021 03:14:49 GMT

Hi @Sarvagna Mahakali There is an easier hack:

a) You can save results locally on the disk and create a hyper link for downloading CSV . You can copy the file to this location: dbfs:/FileStore/table1_good_2020_12_18_07_07_19.csv

b) Then download with link: HTTPS://<yourworkspace>.cloud.databricks.com/files/trace_good_2020_12_18_07_07_19.csv

Hope it helps! 😊