topic Re: Error on ANALYZE TABLE [...] COMPUTE STATISTICS FOR COLUMNS [...] in Data Engineering

Error on ANALYZE TABLE [...] COMPUTE STATISTICS FOR COLUMNS [...]

leungi — Tue, 09 Jul 2024 14:51:30 GMT

End goal is to apply OPTIMIZE and ZORDER table.

However, one of the columns to be ZORDER doesn't have stats collected.

Running ANALYZE generates the error below.

Query

ANALYZE TABLE <catalog>.<schema>.<table> COMPUTE STATISTICS FOR COLUMNS my_col_1, my_col_2;

Error

[INTERNAL_ERROR] Eagerly executed command failed. You hit a bug in Spark or the Spark plugins you use. Please, report this bug to the corresponding communities or vendors, and provide the full stack trace. SQLSTATE: XX000

Re: Error on ANALYZE TABLE [...] COMPUTE STATISTICS FOR COLUMNS [...]

leungi — Thu, 11 Jul 2024 16:19:46 GMT

@Retired_mod , thanks for the detailed suggestions.

1. Running SHOW COLUMN STATS yields the following error:

[PARSE_SYNTAX_ERROR] Syntax error at or near 'CHECK'. SQLSTATE: 42601 line 1, pos 0

== SQL ==
CHECK COLUMN STATS <catalog>.<schema>.<table>
^^^

Ran DESCRIBE DETAIL <catalog>.<schema>.<table> successfully, and it indicates no column stats are currently being collected.

3. Ran ANALYZE TABLE <catalog>.<schema>.<table> COMPUTE STATISTICS FOR COLUMN my_col_1; for a single column, and faced the same error.

Any further troubleshooting tips are much appreciated!