topic Re: AnalysisException: is not a Delta table. but that table is Delta table in Data Engineering

AnalysisException: is not a Delta table. but that table is Delta table

modest2va — Wed, 04 Sep 2024 09:03:56 GMT

When running a Databricks notebook,
an error occurs stating that SOME_TABLE is not a Delta table.

However, after executing the describe detail command and checking the format,
the table is shown as Delta.

Without taking any specific actions, re-running the notebook resolves the issue,
and the notebook operates normally.

Executed cluster:
DBR 13.3 LTS
Spark 3.4.1
Scala 2.12

Has anyone experienced this issue before?
If you have a fundamental solution to the problem, please share it.

Thank you guys in advance.

Re: AnalysisException: is not a Delta table. but that table is Delta table

Witold — Wed, 04 Sep 2024 09:41:39 GMT

Can you share some code and error messages with us?

Re: AnalysisException: is not a Delta table. but that table is Delta table

modest2va — Wed, 04 Sep 2024 10:02:28 GMT

Re: AnalysisException: is not a Delta table. but that table is Delta table

modest2va — Wed, 04 Sep 2024 10:02:02 GMT

Due to security reasons, I can't share the entire code,
but a function, I'm trying to load and process a table using the following code:

map_df = DeltaTable.forName(spark, schema_table).toDF()

The schema_table variable contains the schema and table name where the error occurs. I've confirmed that there's no typo in the schema or table name, so the issue doesn't seem to be related to that.

Error Messages ▼:

Thank you Witold

Re: AnalysisException: is not a Delta table. but that table is Delta table

Witold — Wed, 04 Sep 2024 10:10:35 GMT

schema_table variable contains the schema and table

forName doesn't expect a schema, of course only a table path.

Re: AnalysisException: is not a Delta table. but that table is Delta table

modest2va — Thu, 05 Sep 2024 00:56:47 GMT

yeah I know that

At the first time, That codes didn't work.
Without doing something, I just run again

then That forName worked.

It's so curious

Re: AnalysisException: is not a Delta table. but that table is Delta table

Witold — Thu, 05 Sep 2024 06:55:39 GMT

Another thing you could check is how the underlying data looks like. Maybe the actual writer of the data, messed it up.