topic Re: PERMISSION_DENIED: Cannot access Spark Connect. when trying to run serverless databricks connec in Get Started Discussions

PERMISSION_DENIED: Cannot access Spark Connect. when trying to run serverless databricks connect

ivan7256 — Fri, 25 Jul 2025 19:24:29 GMT

I am not able to run a file as "run as workflow" nor "run with databricks connect" when I choose serverless run on my paid account. However I can perform this action in my free edition account .

See error :

pyspark.errors.exceptions.connect.SparkConnectGrpcException: PERMISSION_DENIED: Cannot access Spark Connect. (requestId=e6437897-ac03-4c06-b7a6-5b25ee676d03)

Re: PERMISSION_DENIED: Cannot access Spark Connect. when trying to run serverless databricks connec

RobTheCoder — Sun, 27 Jul 2025 00:11:05 GMT

I'm having the same problem. I'm admin on both the account and the workspace. I get this with `databricks-connect test` too. I've tried updating my databricks-connect version to `==16.4.*` which seems to be the latest version with serverless support.

One thing I have noticed is that even though I can use serverless in the web UI for notebooks and SQL warehouse, I don't see the serverless option under feature enablement in the account settings. I suspect my account is misconfigured so I'm contacting support about that.

Re: PERMISSION_DENIED: Cannot access Spark Connect. when trying to run serverless databricks connec

SP_6721 — Mon, 28 Jul 2025 13:25:52 GMT

Hi @ivan7256 ,

This might be because serverless compute isn't enabled for workflows in your paid workspace.

Re: PERMISSION_DENIED: Cannot access Spark Connect. when trying to run serverless databricks connec

RobTheCoder — Sat, 02 Aug 2025 18:23:45 GMT

I think adding this to my databricks.yml file helped. I probably did have a permissions problem and just granting these permissions over the whole of the bundle solved my issue.

permissions:
- group_name: my-admin-group
level: CAN_MANAGE
- group_name: my-user-group
level: CAN_VIEW