cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
cancel
Showing results for 
Search instead for 
Did you mean: 

UDF/UDAF/SQL Function not working in Unity catalog Error in DBR 10.4 LTS Shared mode

parthsalvi
Contributor

We're trying to use pandas_udf in DBR 10.4 LTS Shared mode but keep running in the below error :

imageAnalysisException: UDF/UDAF/SQL functions is not supported in Unity Catalog;

But in Single User mode above code works correctly.

3 REPLIES 3

Hubert-Dudek
Esteemed Contributor III

For me, it is like a bug as, in my opinion, that code should be independent of the unity catalog as it sparks code, and you don't register anything in the unity catalog. So please pass it to your databricks representative or as a support ticket.

There are SQL functions (udfs) in Unity, and soon there will be Python one, but it is about ones that are registered in meatstore/catalog.

Maybe the problem here is that you use a shared version, so it wants to register it so it will be available for other users?

Yes, the issue appears only in Shared Mode so we're currently using Single user mode until we get some resolution from databricks.. Previously, we ran into similar issue while broadcasting udf and we got a suggestion from databricks rep to try pandas_udf instead, but that too throws same error in Shared mode.

ngs
New Contributor II

Did Databricks come up with a resolution for your use-case? We're currently facing the same issue.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.