Hi Databricks Support,

We need help confirming whether this is a known DBR 15.4 LTS bug or an unsupported/configuration-specific behavior.

Summary

On a Databricks Runtime 15.4.40 Photon job cluster, `ANALYZE TABLE <catalog>.<schema>.<table> COMPUTE STATISTICS FOR ALL COLUMNS` completes successfully but does not update table-level or column-level optimizer statistics for a Unity Catalog managed Delta table.

The same SQL run from a DBSQL Serverless warehouse works correctly and refreshes the column stats immediately.

Environment

- Runtime: Databricks Runtime 15.4.40 LTS
- Engine: Photon
- Cluster type: job cluster
- Table type: Unity Catalog managed Delta table
- Table properties:
- `delta.universalFormat.enabledFormats = iceberg`
- `delta.enableIcebergCompatV2 = true`
- `delta.columnMapping.mode = name`
- `delta.dataSkippingNumIndexedCols = 50`
- `delta.checkpointPolicy = classic`
- `delta.enableDeletionVectors = false`
- Predictive Optimization: disabled/inherited disabled at catalog level

Affected table

Example table:

`stg_silver_internal.payment.payment_merchant_read_merchant_preauth_transaction`

Reproduction

In the same DBR 15.4.40 Photon job-cluster session:

Run a silver job with 'ANALYZE TABLE ... COMPUTE STATISTICS FOR ALL COLUMNS' but no use.

Comparison
Running the same ANALYZE TABLE ... COMPUTE STATISTICS FOR ALL COLUMNS statement from a DBSQL Serverless warehouse works correctly. The DESCRIBE TABLE EXTENDED <table> <column> stats refresh immediately afterward.

Things already tried
Fully qualified three-part table name
USE CATALOG <catalog>
REFRESH TABLE <table>; result says refresh was not needed
Toggling spark.databricks.delta.uniform.iceberg.sync.convert.enabled
Verifying the same session sees the actual latest data via SELECT max(esupdatedat)
None of these changed the DBR job-cluster behavior.

Questions
Is this a known issue in DBR 15.4 LTS for UC managed Delta tables with UniForm/Iceberg compatibility and column mapping?
Is ANALYZE TABLE ... COMPUTE STATISTICS FOR ALL COLUMNS expected to work from an all-purpose/job cluster in this table configuration?
Is the supported workaround to run optimizer-stat collection from DBSQL Serverless / SQL warehouse instead of DBR job clusters?
Should we use ANALYZE TABLE ... COMPUTE DELTA STATISTICS in addition to regular COMPUTE STATISTICS, or is that only for Delta file-skipping stats and not a replacement for CBO column stats?
Is this fixed in a newer DBR runtime such as DBR 16.x?
Please advise the recommended production-safe workaround and whether this should be treated as a DBR 15.4 LTS bug.

Thanks.