This error is most often due to missing permissions, workspace types, or cluster configuration settings necessary for AI agents and custom tools to access Unity Catalog (UC) functions using Spark Connect in Databricks. The main causes and solutions are detailed below.
Common Causes
-
Your workspace might not be set up as a Serverless workspace, which is required for agents to execute UC functions as tools.
-
The user account running DatabricksFunctionClient does not have explicit access (USAGE permission) to the relevant catalog/schema, or EXECUTE permission for the relevant functions in Unity Catalog.
-
You are using a cluster type or Databricks Runtime version that does not support required permissions for Unity Catalog, such as single-user clusters with older runtimes.
-
The group or user running the notebook does not have workspace access entitlements in Databricks.
Solutions
-
Serverless Workspace Setup: Ensure the Databricks workspace is set up as Serverless before attempting to register tools or run agents.
-
Granting Catalog and Schema Permissions: Explicitly grant USAGE, EXECUTE, and MANAGE permissions on the relevant catalog and schema to your user email or group using SQL GRANT statements. Example:
GRANT USAGE ON CATALOG `your_catalog` TO `<your-email>`;
GRANT USAGE ON SCHEMA `your_schema` TO `<your-email>`;
GRANT EXECUTE ON FUNCTION `your_function` TO `<your-email>`;
-
Cluster and Runtime Configuration: Use a shared cluster, serverless compute, or update the Databricks Runtime version to 15.4 or above. For single-user clusters, upgrade to the required runtime.
-
Workspace Access Entitlements: Make sure your user or group has the Workspace access entitlement enabled in Databricks workspace settings.
If changing the workspace to Serverless or granting permissions does not resolve the SparkConnectGrpcException: PERMISSION_DENIED error, double-check your cluster configuration and role-based access control, then retry with updated settings and entitlements.
These steps should resolve common permission denied issues when registering and invoking Databricks AI agent tools with Unity Catalog.
