Photon plan invariant violated Error

[Vishwanath_Rao]

We've run into a niche error where we get the below message only on our non prod environment, with the same data, with the same code as our prod environment.

org.apache.spark.sql.execution.adaptive.InvalidAQEPlanException: Photon plan invariant violated: Broadcast joins have incompatible build plans regardingPhotonized vs. not-Photonized: ArrayBuffer(BroadcastHashJoin [COLUMN1#1372320], [COLUMN2#1422501], LeftOuter, BuildRight, false

Unsure as to what might be the issue, I've checked the spark logs, and it gives nothing on it as well, there doesn't seem to be any docs on these either. Any help is appreciated.

[Vishwanath_Rao]

Thank you @Retired_mod! It looks like the issue was with a recent release on the databricks end, I'd raised a support ticket just to be sure, the difference also died down when we set these two up, at the cluster level.

spark.conf.set("spark.sql.adaptive.enabled", "true")

spark.conf.set("spark.sql.optimizer.excludedRules", "")

[JAC2703]

Hey, did you raise a ticket and get a resolution to this?