Hi Mich,

I have a similar pandas_udf. The scripts failed to run on an all purpose cluster. The error is [UC_COMMAND_NOT_SUPPORTED.WITHOUT_RECOMMENDATION] The command(s): Spark higher-order functions are not supported in Unity Catalog. Do you know by any chance how to make it work with UC or it is not supported by the UC? I couldn't find the relevant documentation. 

Thanks!