topic Re: Managed Delta Table corrupted in Data Engineering

Managed Delta Table corrupted

Rafael-Sousa — Fri, 07 Feb 2025 14:17:52 GMT

Hey guys,

Recently, we have add some properties to our delta table and after that, the table shows error and we cannot do anything. The error is that: (java.util.NoSuchElementException) key not found: spark.sql.statistics.totalSize

I think maybe this is a bug related to spark connect client in databricks.

Re: Managed Delta Table corrupted

Alberto_Umana — Fri, 07 Feb 2025 14:26:01 GMT

Hi @Rafael-Sousa,

This issue can occur when certain statistics are missing or not properly updated in the Delta table's metadata.

You can try to manually update the statistics for the Delta table. This can be done by running the ANALYZE TABLE command to compute statistics for the table:

ANALYZE TABLE your_table_name COMPUTE STATISTICS;

Re: Managed Delta Table corrupted

Rafael-Sousa — Fri, 07 Feb 2025 14:53:45 GMT

Hey @Alberto_Umana ,

The problem is that I am not able to run this commando too. The error persists for any kind of operation that I try to run on the table.

Re: Managed Delta Table corrupted

Alberto_Umana — Fri, 07 Feb 2025 17:06:24 GMT

Hi @Rafael-Sousa,

Could you please raise a support case for this, to investigate this further?

help@databricks.com