topic Re: Location not empty but not a Delta table in Data Engineering

Location not empty but not a Delta table

Distributed_Com — Mon, 24 Oct 2022 18:41:31 GMT

I need help or insight regarding the following errors. My instructors (Brooke Wenig with Conor Murphy) ran this code successfully on our course video, but I cannot replicate what she did.

Here is the code and below it is the outcome from my Coursera course, workspace section 4.5 notebook:

CREATE DATABASE IF NOT EXISTS Databricks;

USE Databricks;

DROP TABLE IF EXISTS fireCallsDelta;

CREATE TABLE fireCallsDelta

USING DELTA

PARTITIONED BY (City)

SELECT * FROM fireCallsParquet

(3) Spark Jobs

Job 3 View(Stages: 1/1)
Job 4 View(Stages: 1/1, 1 skipped)
Job 5 View(Stages: 1/1, 2 skipped)

Error in SQL statement: AnalysisException: Cannot create table ('`spark_catalog`.`databricks`.`fireCallsDelta`'). The associated location ('dbfs:/user/hive/warehouse/databricks.db/firecallsdelta') is not empty but it's not a Delta table

Re: Location not empty but not a Delta table

dennyglee — Tue, 25 Oct 2022 04:56:56 GMT

The `fireCallsDelta` table already exists, and it appears it was created as a PARQUET table. A couple of options:

DELETE the original Parquet table as a separate statement (i.e., in a separate cell so it can complete itself before any other jobs executing)
Rename the table to something else if perhaps others are running the same demo.

Re: Location not empty but not a Delta table

Distributed_Com — Tue, 25 Oct 2022 13:53:47 GMT

Thanks. Renaming table as you suggested allowed the query to run successfully.

Re: Location not empty but not a Delta table

Distributed_Com — Tue, 25 Oct 2022 13:54:34 GMT

Done. Thanks.

Re: Location not empty but not a Delta table

gilo12 — Mon, 10 Jul 2023 17:44:20 GMT

DELETE the original Parquet table as a separate statement

How can this be done? simple query "DROP TABLE .... " still failing with "cannot be found"