AI Playground access Tools(SQL Function) with spark connect PERMISSION_DENIED

loujiang
New Contributor

yesterday

Hello, Dear community, 

I have one SQL function defined in Unity Catalog accessing delta table in the unity catalog. It works normally if i use SQL to execute Function. 

I see the possibility connecting function to Agent/ Chat. For the quick prototype, I just try with the AI playground tab and include the function as a tool. With the first trial, it give back the Error: dbconnectshaded.v15.org.sparkproject.io.grpc.StatusRuntimeException: PERMISSION_DENIED: PERMISSION_DENIED: Cannot access Spark Connect. (requestId=fd346bc6-eafb-42eb-a32e-883578db6d1b)Error: Error: dbconnectshaded.v15.org.sparkproject.io.grpc.StatusRuntimeException: PERMISSION_DENIED: PERMISSION_DENIED: Cannot access Spark Connect. (requestId=fd346bc6-eafb-42eb-a32e-883578db6d1b). 

Seems like the Tool and Function call are different. How can I enable the Tool getting the spark and how it will be authenticated? For my simple case, I do not what to use the heavy solution with link to genie spaces. 

Thanks 

best wishes

loujiang

0 Kudos

Oliver_learning
New Contributor III

6 hours ago

Hello @loujiang,  a few questions to understand the issue.

Does your workspace have serverless compute enabled?
When AI Playground calls your function as a tool, is there an active classic cluster attached ?
If you have a classic cluster, is it configured with Spark Connect enabled (which runtime, single-user or shared access mode)?
Is it a scalar function or a table-valued function?
Are you running as your personal user or through a service principal?
Does your user have USE CATALOG, USE SCHEMA, and EXECUTE on the function ?


Why I'm asking :
calling a SQL function directly via SQL uses a SQL warehouse.
But when AI Playground invokes it as a tool, it goes through Spark Connect — a completely different entry point that needs its own compute and auth.

Try 

spark.sql("SELECT * from catalog.shema.my_function(arg)") in a notebook.
 

 

View solution in original post

0 Kudos

loujiang
New Contributor
In response to Oliver_learning

4 hours ago

Thanks for the hint. My problem is at AI playground and it goes different entry point. I have to enable the serverless generic compute to enable this usage. The normal function works at the serverless SQL. Maybe do you have some documentation explain the authentication / connection process happened behind?  

0 Kudos