Databricks Community

We've run into a niche error where we get the below message only on our non prod environment, with the same data, with the same code as our prod environment.org.apache.spark.sql.execution.adaptive.InvalidAQEPlanException: Photon plan invariant violat...

We're in the middle of migrating to Databricks and found that the same path on s3 is producing different counts between EMR (Spark 2.4.4) and Databricks (Spark 3.4.1) it is a simple spark.read.parquet().count(), tried multiple solutions like making t...