Databricks Community

Sainath368 · 09-01-2025

Hi everyone,We’re currently reviewing the performance impact of enabling Photon acceleration on our Databricks jobs, particularly those involving table maintenance tasks. Our job includes three main operations: OPTIMIZE, VACUUM, and ANALYZE_COMPUTE_S...

Sainath368 · 08-04-2025

Hi everyone,I regularly (once a week) run the analyze table compute statistics command on all my tables in Databricks to keep statistics up to date for query optimization.In the Spark table UI catalog, I can see some statistics metadata like spark.st...

Sainath368 · 07-28-2025

Hi all,I need to run weekly maintenance on approximately 7,000 tables in my Databricks environment, involving OPTIMIZE, VACUUM, and ANALYZE TABLE (for statistics calculation) on all tables.My question is: between the Ev4, Edv4, and Fsv2 VM series, wh...

Sainath368 · 07-15-2025

Hi, I want some clarification regarding running ANALYZE TABLE <table_name> COMPUTE STATISTICS. Can anyone please help me understand if this command will throw errors or cause issues while data is loading into the table at the time of execution? Any i...

Sainath368 · 07-08-2025

I ran the command ANALYZE TABLE <TABLE_NAME> COMPUTE STATISTICS FOR COLUMNS 'COL1,COL2' to compute statistics for specific columns, but I noticed that the distinct counts returned by this command differ from the results obtained by running a direct C...

Sainath368 · 07-08-2025

Hi @Waldar , The approximate count matches the result from the DESCRIBE EXTENDED query. This confirms that the query you provided helps verify the count correctly. But what would be the reason?

Sainath368 · 07-04-2025

Thank you for your earlier reply—which was helpful—but I’m still looking for a clearer explanation, so I’m asking this in more detail.I'm working with external Delta tables that don't use Predictive I/O optimization, so I set the delta.dataSkippingSt...

Sainath368 · 07-03-2025

I ran ANALYZE TABLE <table_name> COMPUTE DELTA STATISTICS on my tables to calculate column-level statistics at the file level for each Parquet file inside the Delta table. Now, I’m wondering which command is better to run next: ANALYZE TABLE <table_n...

Sainath368 · 07-02-2025

I ran ANALYZE TABLE <table_name> COMPUTE DELTA STATISTICS on my tables to calculate column-level statistics at the file level for each Parquet file inside the Delta table. Now, I’m wondering which command is better to run next: ANALYZE TABLE <table_n...

Databricks Community

User Stats

User Activity

Is Photon Acceleration Helpful for All Maintenance Tasks (OPTIMIZE, VACUUM, ANALYZE_COMPUTE_STATS)?

How to Retrieve the spark.statistics.createdAt When Statistics Were Last Updated in Databricks?

E series vs F series VM's

ANALYZE TABLE <table_name> COMPUTE STATISTICS- Data loading

ANALYZE command for stats collection - distinct_count difference for a column

Re: ANALYZE command for stats collection - distinct_count difference for a column

Re: Clarification Needed: COMPUTE STATISTICS vs COMPUTE DELTA STATISTICS on Delta Tables

Re: Clarification Needed: COMPUTE STATISTICS vs COMPUTE DELTA STATISTICS on Delta Tables

Re: Clarification Needed: COMPUTE STATISTICS vs COMPUTE DELTA STATISTICS on Delta Tables