topic Re: Unity Credential Scope id not found in thread locals in Data Engineering

Unity Credential Scope id not found in thread locals

kasiviss42 — Mon, 30 Dec 2024 11:34:03 GMT

i am facing issue :- [UNITY_CREDENTIAL_SCOPE_MISSING_SCOPE] Missing Credential Scope. Unity Credential Scope id not found in thread locals.

Issue occurs:-

when we try to list files using dbutils.fs.ls

and also this occurs at times when we try to write or read a parquet file

This occurs 4 in 10 times a day. This occurs in prod as well .

No Proper fix has been Documented.

Can someone help here ? This is blocked since many days

Re: Unity Credential Scope id not found in thread locals

Alberto_Umana — Mon, 30 Dec 2024 13:03:00 GMT

Hi @kasiviss42,

Are you using any Scala code in your notebook?

The error [UNITY_CREDENTIAL_SCOPE_MISSING_SCOPE] Missing Credential Scope. Unity Credential Scope id not found in thread locals that you are encountering when using dbutils.fs.ls and while reading or writing parquet files is a known issue related to Unity Catalog in Databricks. This issue occurs because Unity Credential Scopes are stored as thread locals, and in certain scenarios, such as when using Scala thread pools, these scopes are not transferred to the new threads, leading to the error

Re: Unity Credential Scope id not found in thread locals

kasiviss42 — Mon, 30 Dec 2024 16:52:51 GMT

Its python

Re: Unity Credential Scope id not found in thread locals

VZLA — Mon, 30 Dec 2024 17:16:50 GMT

Hi @kasiviss42, Thank you for sharing the error details!

To provide better guidance, could you clarify the following details about your environment and use case?

Cluster Configuration:
- Are you using a single-user cluster or a shared/multi-user cluster? If single-user, have you set spark.databricks.unityCatalog.singleUserAutoGlobalScope to true?
Workload Details:
- What type of operation or workload are you running when this error occurs (e.g., a Spark job, Delta Lake operation, or background process)?
- Are you using thread pools, asynchronous tasks, or submitting jobs programmatically?
Specific Context:
- Does this issue occur consistently or intermittently?
- Have you noticed if certain operations or configurations (e.g., checkpointing, writing to S3, or specific thread usage) trigger the error?
Setup and Logs:
- Are you running this on a free trial or a production workspace?
- Can you share the specific stack trace or log details (excluding sensitive information) to help identify where the issue arises?

Re: Unity Credential Scope id not found in thread locals

kasiviss42 — Mon, 30 Dec 2024 17:51:57 GMT

1. Cluster Configuration

Policy :- Shared Compute

Access Mode :- Shared

Hence we haven't set singleUserAutoGlobalScope to True.

WorkLoad Details.

Executing a notebook written in Pyspark .

i. We are simply writing dataframe to parquet and read it back.(For kind of checkpointing)

ii. And using dbutils.fs.ls through volumes

Above two cases error is thrown intermittently

We are not using asynchronous threads or pools

Error is frequently occurring i.e. 40% of the times it fails . 60% it succeeds. We need a code to be 100% succeeding

We have noticed failures when dbutils.fs.ls is being executed in the code. as well as when we write dataframe to parquet or read the dataframe written in parquet back

We are running on a prod workspace.

i. Below error occurs when a parquet file is written or read

Below is the underlying error caused by :- (com.databricks.backend.daemon.data.common.InvalidMountException)

Caused by: com.databricks.unity.error.MissingCredentialScopeException: [UNITY_CREDENTIAL_SCOPE_MISSING_SCOPE] Missing Credential Scope. Unity Credential Scope id not found in thread locals.. SQLSTATE: XXKUC

ii. Below error occurs when dbutils.fs.ls is being executed.

Exception: An error occurred while calling o542.ls.
: com.databricks.backend.daemon.data.common.InvalidMountException:

Strange thing here is both the code is same notebook .

One time it fails at parquet read but succeeds dbutils.fs.ls

other time it fails at dbutils.fs.ls itself .

Please suggest

Re: Unity Credential Scope id not found in thread locals

VZLA — Mon, 30 Dec 2024 20:31:55 GMT

@kasiviss42 thanks for sharing these details.

We can potentially get to the root cause by enabling the traceLogging for UC, but the communication will not be efficient through this channel. Could you please raise a Support Ticket? We'd like to help you address this problem or at the very least find a workaround.

Re: Unity Credential Scope id not found in thread locals

ashishCh — Fri, 24 Jan 2025 14:26:34 GMT

Any solution for this, I am facing this too when creating a thread and using SQL query inside it

Re: Unity Credential Scope id not found in thread locals

kasiviss42 — Fri, 24 Jan 2025 16:27:51 GMT

Use dbr 15.4 lts and try once

Re: Unity Credential Scope id not found in thread locals

ashishCh — Fri, 24 Jan 2025 17:02:53 GMT

Thanks for the reply.
Its working in dbr 15.4 but I want to use it with 13.3, is there a workaround?

Re: Unity Credential Scope id not found in thread locals

kasiviss42 — Fri, 24 Jan 2025 17:40:02 GMT

Databricks has released the fix in DBR 15.4 LTS

not sure if there is any work around for < 15.4 versions

Re: Unity Credential Scope id not found in thread locals

ashishCh — Fri, 24 Jan 2025 17:51:42 GMT

Thanks for the information.
Can you share me the link where they have mentioned that this has been fixed.