topic Re: AnalysisException: is not a Delta table. in Data Engineering

AnalysisException: is not a Delta table.

joselita — Thu, 13 Oct 2022 10:55:02 GMT

Hello,

I changed the DBR from 7.2 to 10.4 and I receive the following error: AnalysisException: is not a Delta table.

The table is create , using DELTA. so for sure is a Delta table, even though, I read that I read that from vers. 8 all tables are Delta as default and don't need to write USING DELTA.

What can say me about this error?

AnalysisException: `default`.`stg_data_load` is not a Delta table.;

---------------------------------------------------------------------------

AnalysisException Traceback (most recent call last)

<command-4355583460198494> in <module>

50 # Fill staging table

---> 51 df.write.insertInto('STG_DATA_LOAD', overwrite = False)

/databricks/spark/python/pyspark/sql/readwriter.py in insertInto(self, tableName, overwrite)

1147 if overwrite is not None:

1148 self.mode("overwrite" if overwrite else "append")

-> 1149 self._jwrite.insertInto(tableName)

1150

1151 def saveAsTable(self, name, format=None, mode=None, partitionBy=None, **options):

/databricks/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py in __call__(self, *args)

1302

1303 answer = self.gateway_client.send_command(command)

-> 1304 return_value = get_return_value(

1305 answer, self.gateway_client, self.target_id, self.name)

Re: AnalysisException: is not a Delta table.

Ryan_Chynoweth — Thu, 13 Oct 2022 14:03:43 GMT

Hi @JOSELITA MOLTISANTI can you run the following commands and share the output?

table_name = "stg_data_load"
path = spark.sql(f"describe detail {table_name}").select("location").collect()[0][0].replace('dbfs:', '')
dbutils.fs.ls(path)

Re: AnalysisException: is not a Delta table.

joselita — Thu, 13 Oct 2022 18:25:22 GMT

Mnt/.../RAW/community/datalog

Re: AnalysisException: is not a Delta table.

Ryan_Chynoweth — Thu, 13 Oct 2022 19:29:11 GMT

Hi @JOSELITA MOLTISANTI, It should look more like the following. I am interested in seeing the contents of the path not the path itself.

Re: AnalysisException: is not a Delta table.

Ryan_Chynoweth — Thu, 13 Oct 2022 19:36:47 GMT

Also, it would be more helpful if you could provide code snippets as well. I see you are trying to do the write.insertInto command.