- last edited by
Rishabh_Tiwari
Hi everyone,
Starting this thread to check if anyone else has seen a similar issue, and sharing the solution because in our case, it was a detail we hadn't noticed it in the documentation.
https://docs.databricks.com/aws/en/compute/single-user-fgac
Problem
Starting on August 26, we began experiencing issues with some jobs/pipelines (running on dedicated "single user" mode) that consumed data from the system table (system.audit.access, with the error below:
After upgrading DBR (Databricks Runtime) version to DBR 15.4 LTS+ (for read queries), the error message switch to this one:
[FGAC_ON_DEDICATED_COMPUTE_FAILED] Fine-grained access control (FGAC) on dedicated compute failed due the following exception: PERMISSION_DENIED: PERMISSION_DENIED: Cannot access Spark Connect. (requestId=cc5287db-4c51-49c8-8ed6-6eeab6a66032)
Cause
Query text will be masked by default in the query history system table, the Query History API, audit logs, and the List Queries API starting August 26, 2026
You are receiving this notice because you have one or more workspaces that read query text from the query history system table, the Query History API, audit logs, or the List Queries API.
As more customers run Databricks for mission-critical and transactional workloads, we are raising the default security posture for every account. Query text often contains PII, including names, email addresses, and account numbers embedded in filter conditions, alongside table names and business logic. It was already possible to protect this before, but it required setup. We are making that protection the default.
Starting August 26, 2026, query text will be masked by default and visible only to account administrators and members of the databricks_pii_access group who also have access to the underlying table or API. This keeps your data secure from inadvertent sharing and prevents agentic systems from gaining unintended access to it.
Resolution
- Keep the dedicated cluster, but make sure it's on DBR 15.4 LTS+ (for read queries) .
Even when using the runtime mentioned in the documentation, we continued to encounter errors; that was when we discovered that Serverless for Jobs/Notebooks needs to be enabled for the workspace so that the FGAC operation would work for jobs in Dedicated ("Single User") security mode:
- Since the mask is delegated to serverless, the workspace needs the serverless-compute path available for the FGAC hop to complete. Note: this is Serverless compute for jobs/notebooks/workflows (workspace admin setting), not the "Can use" permission on a serverless SQL warehouse. Those are different settings; the one FGAC needs is the jobs/workflows serverless entitlement.
- Standard clusters (Shared) evaluate FGAC natively, with no delegation hop, so the masked system tables read fine. Trade-off: Standard mode doesn't support a few low-level APIs, so the job needs a small code change to drop direct SparkContext/RDD usage and in-code SparkConf/spark.conf mutations (use the DataFrame/SQL APIs and set Spark configs at the cluster level instead).
- How to Enable it?
Account Console
The feature you need is Serverless compute for jobs/notebooks/workflows, enabled at the account level by an account admin (Trade-off is that enables for all workspace in the account, so, you need to assess the risks and potential cost impacts.):
- Login in to https://accounts.azuredatabricks.net/
- Account Console → Settings → Feature enablement → "Serverless compute" → enable (accepting the serverless terms if prompted).
Workspace Console
I found this option in the "Previews" section at workspace level and enabled it:
Did you run into a similar issue? What did you think of this new feature?
How are you managing or limiting Serverless costs for jobs and workflows?
Regards,
Danilo Van der Linden
