Werners, thanks again from getting in touch.

I managed to fix this by replacing cross apply with cross join.

However, I'm not getting the error:

Undefined function: iif. This function is neither a built-in/temporary function, nor a persistent function that is qualified as spark_catalog.default.iif.

Is there a equivalent method to iif in Databricks