cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Upload file to DBFS fails with error code 0

hravilla
New Contributor

When trying to upload to DBFS from local machine getting error as "Error occurred when processing file ... : Server responded with 0 code"

DBR 7.3 LTS

Spark 3.0.1

Scala 2.12

Uploading the file using the "upload" in the Databricks cloud console, the cp through Databricks-cli does not respond.

2 REPLIES 2

PramodNaik
New Contributor II

Even I am facing the same issue with GCP databricks. I am able to upload files with smaller size. When i tried with 3MB file, databricks chokes. I get the above error.

I tried with aws databricks, it works good even for bigger size files.

LokeshManne
New Contributor III

@PramodNaik

The error you are facing not because of file size but because the file you are trying to upload contains PII or SPII data  words like dob, Token, accesskey, password..,etc.

LokeshManne_1-1747218087347.png

Solution: Rename such data like date_of_birth, token_no, access_key, default_password,..etc. It will upload now.

If you're not able to find the data which is restricted by databricks policy and throwing above exception, you can cut the file half and try to upload in first half of the set, But before that initially you try to upload with single row of data, if it fails the columns contain the PII/SPII data, if it succeed which means the rows contain PII/SPII data so now try uploading incrementally untill the databricks throws the error. you try to upload with first half set of data if it throws an error, which mean there is an PII/SPII data in the fist half and again split the first half into half and upload, if it throws an error then again do the same untill no error, Case 1: the row where the error throws is the data you need to update. Case 2: when the part o the file uploads successfully, which means there is no error in that part(fist half or second half)

Lokesh Manne