Uploading local file

[Shoaib8587]

 

Since, last two day i getting an error called 

"ERROR OCCURRED WHEN PROCESSING FILE:[OBJECT OBJECT]" 

While uploading any "csv" or "json" file from my local system but it shows or running my previous file but give error after uploading a new file

[MJ_BE8]

I got the same error today too when trying to upload .csv file. This error never appeared before. 

[PraveenReddy21]

I am also getting same  issue last 2  days onwards  , may be owner restricted  .

[Shoaib8587]

Yeah! I also think that it's a technical issue because it happened from past 2 days.

[Fahadra02]

I am getting the same issue 😕 Is there any other way?

[Mansi-DataEng]

Hello,
Yes, I am also getting the same error while uploading csv of 689KB.

[Fahadra02]

Do anybody know any alternative for learning pyspark?

[Shoaib8587]

Yes, you can use "PW skill lab" it free open source to use.

[jefflipkowitz]

Is this for the community edition only your seeing this issue?

[Shoaib8587]

Yes i make a account on Databricks community.

[Fahadra02]

Anybody tried using databricks without community version? If it's working fine?

[Pruthviraj · Accepted solution]

Hi,

This has been fixed now, 

[LokeshManne]

If you are using databricks community edition, the error you are facing is because the file you are trying to upload contains PII or SPII data ( Personally Identifiable Information OR Sensitive Personally Identifiable Information)  words like dob, Token, accesskey, password..,etc.

Solution: Rename such data like date_of_birth, token_no, access_key, default_password,..etc. It will upload now.

If you're not able to find the data which is restricted by databricks policy and throwing above exception, you can cut the file half and try to upload in first half of the set, But before that initially you try to upload with single row of data, if it fails the columns contain the PII/SPII data, if it succeed which means the rows contain PII/SPII data so now try uploading incrementally untill the databricks throws the error. you try to upload with first half set of data if it throws an error, which mean there is an PII/SPII data in the fist half and again split the first half into half and upload, if it throws an error then again do the same untill no error, Case 1: the row where the error throws is the data you need to update. Case 2: when the part of the file uploads successfully, which means there is no error in that part(fist half or second half)