UDF/UDAF/SQL Function not working in Unity catalog Error in DBR 10.4 LTS Shared mode

[parthsalvi]

We're trying to use pandas_udf in DBR 10.4 LTS Shared mode but keep running in the below error :

AnalysisException: UDF/UDAF/SQL functions is not supported in Unity Catalog;

But in Single User mode above code works correctly.

[Hubert-Dudek]

For me, it is like a bug as, in my opinion, that code should be independent of the unity catalog as it sparks code, and you don't register anything in the unity catalog. So please pass it to your databricks representative or as a support ticket.

There are SQL functions (udfs) in Unity, and soon there will be Python one, but it is about ones that are registered in meatstore/catalog.

Maybe the problem here is that you use a shared version, so it wants to register it so it will be available for other users?

[parthsalvi]

Yes, the issue appears only in Shared Mode so we're currently using Single user mode until we get some resolution from databricks.. Previously, we ran into similar issue while broadcasting udf and we got a suggestion from databricks rep to try pandas_udf instead, but that too throws same error in Shared mode.

[ngs]

Did Databricks come up with a resolution for your use-case? We're currently facing the same issue.