FileBug in DBFS? Can not remove file (table) nor create it in Apache Spark (TM) SQL for Data Analysts Coursera course from Unit 6.2 onwards on.

Ougagagoubu — Sat, 18 Sep 2021 20:06:38 GMT

Hello,

as the title already suggests, i'm not able to remove a file via the shell (%sh rm -f "path") nor continue the notebook 6.2 onwards on (6.3 etc...) inside DataBricks. I'm using the DataBricks Community edition.

While the error message is clear:

"Error in SQL statement: AnalysisException: Can not create the managed table('`default`.`DeviceData`'). The associated location('dbfs:/user/hive/warehouse/devicedata') already exists.",

I am not able to figure out how to resolve this. Skipping to the next cells will yield:

"Error in SQL statement: AnalysisException: Table or view not found for 'DESCRIBE TABLE': DeviceData; line 1 pos 0"

i tried removing it via the shell but it has no effect. I tried restart and clearing all clusters - no effect. I'm not familiar with the Classroom setup, so i could not find a solution there either. Also the table doesn't show up under "Data" via the Databricksexplorer.

Thank you in advance.

Best regards

topic FileBug in DBFS? Can not remove file (table) nor create it in Apache Spark (TM) SQL for Data Analysts Coursera course from Unit 6.2 onwards on. in Data Engineering

FileBug in DBFS? Can not remove file (table) nor create it in Apache Spark (TM) SQL for Data Analysts Coursera course from Unit 6.2 onwards on.