Query failures are seen during the TPC-DS performance benchmark run

Data Engineering

Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.

When I'm running TPC-DS (1TB) benchmark on Photon 10.2 and I see the following failures:

Queries Q06, Q09 and Q41 fail with the error "Query: AEValueSubQuery is not supported".

Q66 fails with the error "[MISSING_COLUMN] org.apache.spark.sql.AnalysisException: Column 'x.xw_warehouse_sq_ft' does not exist" and

Q87 fails with "Query: AEExcept is not supported." error.

My understanding is that we should be able to run TPC-DS without any modification of the queries. Any idea why I'm seeing errors and how I can overcome these errors?

I tried to upgrade to Photon 10.4 and I still see the same errors.

Appreciate any help.

Thanks

Suresh