Upload file to DBFS fails with error code 0
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-19-2021 07:20 AM
When trying to upload to DBFS from local machine getting error as "Error occurred when processing file ... : Server responded with 0 code"
DBR 7.3 LTS
Spark 3.0.1Scala 2.12
Uploading the file using the "upload" in the Databricks cloud console, the cp through Databricks-cli does not respond.
- Labels:
-
Dbfs - databricks file system
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-22-2021 12:20 AM
Even I am facing the same issue with GCP databricks. I am able to upload files with smaller size. When i tried with 3MB file, databricks chokes. I get the above error.
I tried with aws databricks, it works good even for bigger size files.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-14-2025 03:21 AM
The error you are facing not because of file size but because the file you are trying to upload contains PII or SPII data words like dob, Token, accesskey, password..,etc.
Solution: Rename such data like date_of_birth, token_no, access_key, default_password,..etc. It will upload now.
If you're not able to find the data which is restricted by databricks policy and throwing above exception, you can cut the file half and try to upload in first half of the set, But before that initially you try to upload with single row of data, if it fails the columns contain the PII/SPII data, if it succeed which means the rows contain PII/SPII data so now try uploading incrementally untill the databricks throws the error. you try to upload with first half set of data if it throws an error, which mean there is an PII/SPII data in the fist half and again split the first half into half and upload, if it throws an error then again do the same untill no error, Case 1: the row where the error throws is the data you need to update. Case 2: when the part o the file uploads successfully, which means there is no error in that part(fist half or second half)