topic Re: UDF/UDAF/SQL Function not working in Unity catalog Error in DBR 10.4 LTS Shared mode in Data Governance

UDF/UDAF/SQL Function not working in Unity catalog Error in DBR 10.4 LTS Shared mode

parthsalvi — Tue, 20 Sep 2022 05:59:48 GMT

We're trying to use pandas_udf in DBR 10.4 LTS Shared mode but keep running in the below error :

AnalysisException: UDF/UDAF/SQL functions is not supported in Unity Catalog;

But in Single User mode above code works correctly.

Re: UDF/UDAF/SQL Function not working in Unity catalog Error in DBR 10.4 LTS Shared mode

Hubert-Dudek — Tue, 20 Sep 2022 10:50:23 GMT

For me, it is like a bug as, in my opinion, that code should be independent of the unity catalog as it sparks code, and you don't register anything in the unity catalog. So please pass it to your databricks representative or as a support ticket.

There are SQL functions (udfs) in Unity, and soon there will be Python one, but it is about ones that are registered in meatstore/catalog.

Maybe the problem here is that you use a shared version, so it wants to register it so it will be available for other users?

Re: UDF/UDAF/SQL Function not working in Unity catalog Error in DBR 10.4 LTS Shared mode

parthsalvi — Wed, 21 Sep 2022 17:08:55 GMT

Yes, the issue appears only in Shared Mode so we're currently using Single user mode until we get some resolution from databricks.. Previously, we ran into similar issue while broadcasting udf and we got a suggestion from databricks rep to try pandas_udf instead, but that too throws same error in Shared mode.

Re: UDF/UDAF/SQL Function not working in Unity catalog Error in DBR 10.4 LTS Shared mode

ngs — Thu, 19 Jan 2023 12:28:46 GMT

Did Databricks come up with a resolution for your use-case? We're currently facing the same issue.