default auth: cannot configure default credentials

SankaraiahNaray — Fri, 18 Oct 2024 10:06:25 GMT

I'm trying to use dbutils from WorkspaceClient and i tried to run this code from databricks notebook.

But i get this error

Error:

ValueError: default auth: cannot configure default credentials

Code:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient()

d = w.dbutils.fs.ls('/')

for f in d:

print(f.path)

Re: default auth: cannot configure default credentials

VZLA — Thu, 31 Oct 2024 17:53:39 GMT

To resolve the ValueError: default auth: cannot configure default credentials error when using dbutils from WorkspaceClient in a Databricks notebook, follow these steps:

Ensure SDK Installation: Make sure the Databricks SDK for Python is installed. You can install it using the %pip magic command in a notebook cell (-U or --upgrade):
1. %pip install -U databricks-sdk
Restart Python: After installing the SDK, restart the Python environment to make the installed library available:
1. dbutils.library.restartPython()
Use Default Notebook Authentication: The Databricks notebook automatically handles authentication. You can directly use the WorkspaceClient without setting environment variables:
1. from databricks.sdk import WorkspaceClient # Initialize WorkspaceClient w = WorkspaceClient() # List files in the specified directory d = w.dbutils.fs.ls('/') for f in d: print(f.path)

By following these steps, you should be able to resolve the authentication issue and run your code successfully within a Databricks notebook.

Some troubleshooting steps that can help you find the problem:

Print the environment variables, look for DATABRICKS_HOST and/or DATABRICKS_TOKEN.

import os for key, value in os.environ.items(): print(f'{key}: {value}')

You may also share the full stacktrace here, I'm assuming there is more printed along with the "ValueError: default auth: cannot configure default credentials". Are you trying this in a Serverless or Classic Cluster? Which DBR release?

topic default auth: cannot configure default credentials in Data Engineering

default auth: cannot configure default credentials

Re: default auth: cannot configure default credentials