I am working on Azure Databricks, with Databricks Runtime version being - 14.3 LTS (includes Apache Spark 3.5.0, Scala 2.12). I am facing the following issue.

Suppose I have a view named v1 and a database f1_processed created from the following command.

CREATE DATABASE IF NOT EXISTS f1_processed
LOCATION "abfss://processed@formula1dl679student.dfs.core.windows.net/"

This is creating a database in the container named processed. Suppose I already have some folder named circuits in that container.

If I run the following command to create a managed table in parquet format from a dataframe in that location using the command below.

circuits_final_df.write.mode("overwrite").format("parquet").saveAsTable("f1_processed.circuits")

It gives an error as follows

SparkRuntimeException: [LOCATION_ALREADY_EXISTS] Cannot name the managed table as 
`spark_catalog`.`f1_processed`.`circuits`, as its associated location 
'abfss://processed@formula1dl679student.dfs.core.windows.net/circuits' already exists. 
Please pick a different table name, or remove the existing location first. SQLSTATE: 42710

However, if I try the same thing in delta format, it runs fine. So the following code runs fine.

circuits_final_df.write.mode("overwrite").format("delta").saveAsTable("f1_processed.circuits")

Also, while creating this delta table, it doesn't remove any files from the folder. It just adds the new files.

Since the result mixes the existing data and new data, it seems it is a bug and it should not happen. Any help is appreciated.