AnalysisException: is not a Delta table. but that table is Delta table
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-04-2024 02:03 AM
When running a Databricks notebook,
an error occurs stating that SOME_TABLE is not a Delta table.
However, after executing the describe detail command and checking the format,
the table is shown as Delta.
Without taking any specific actions, re-running the notebook resolves the issue,
and the notebook operates normally.
Executed cluster:
DBR 13.3 LTS
Spark 3.4.1
Scala 2.12
Has anyone experienced this issue before?
If you have a fundamental solution to the problem, please share it.
Thank you guys in advance.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-04-2024 02:41 AM
Can you share some code and error messages with us?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-04-2024 03:02 AM
Due to security reasons, I can't share the entire code,
but a function, I'm trying to load and process a table using the following code:
map_df = DeltaTable.forName(spark, schema_table).toDF()
The schema_table variable contains the schema and table name where the error occurs. I've confirmed that there's no typo in the schema or table name, so the issue doesn't seem to be related to that.
Error Messages ▼:
Thank you Witold
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-04-2024 03:00 AM - edited 09-04-2024 03:02 AM
--
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-04-2024 03:10 AM
schema_table variable contains the schema and table
forName doesn't expect a schema, of course only a table path.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-04-2024 05:56 PM
yeah I know that
At the first time, That codes didn't work.
Without doing something, I just run again
then That forName worked.
It's so curious
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-04-2024 11:55 PM
Another thing you could check is how the underlying data looks like. Maybe the actual writer of the data, messed it up.