cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Managed Delta Table corrupted

Rafael-Sousa
Contributor II

Hey guys,

Recently, we have add some properties to our delta table and after that, the table shows error and we cannot do anything. The error is that: (java.util.NoSuchElementException) key not found: spark.sql.statistics.totalSize

I think maybe this is a bug related to spark connect client in databricks.

Rafael Sousa
3 REPLIES 3

Alberto_Umana
Databricks Employee
Databricks Employee

Hi @Rafael-Sousa,

This issue can occur when certain statistics are missing or not properly updated in the Delta table's metadata.

You can try to manually update the statistics for the Delta table. This can be done by running the ANALYZE TABLE command to compute statistics for the table:

 

ANALYZE TABLE your_table_name COMPUTE STATISTICS;
 

Rafael-Sousa
Contributor II

Hey @Alberto_Umana ,

The problem is that I am not able to run this commando too. The error persists for any kind of operation that I try to run on the table.

Rafael Sousa

Alberto_Umana
Databricks Employee
Databricks Employee

Hi @Rafael-Sousa,

Could you please raise a support case for this, to investigate this further?

help@databricks.com

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group