topic Databricks fs ls does not work in Data Engineering

Databricks fs ls does not work

Suresh_AWS — Fri, 05 May 2023 20:43:17 GMT

I have configured the databricks CLI using token generated in Azure

databricks configure --aad-token

Configure is success.

When I used the command "databricks fs ls" I am running into an error as shown below

Error: AttributeError: type object 'Retry' has no attribute 'DEFAULT_METHOD_WHITELIST'

Can somebody point to me how to resolve this issue

Re: Databricks fs ls does not work

karthik_p — Fri, 05 May 2023 21:02:34 GMT

@Venkata Suresh Chavala can you try to test by providing dbfs at the end

databricks fs ls dbfs:/

it shows what directories exists in dbfs

Re: Databricks fs ls does not work

Suresh_AWS — Fri, 05 May 2023 21:05:06 GMT

Hi @karthik p ,

I got the same error. I am running on Mac

I ran in debug mode and here is the trace

databricks fs ls dbfs:/ --debug
 
HTTP debugging enabled
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/databricks_cli/utils.py", line 45, in decorator
    return function(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/databricks_cli/configure/config.py", line 65, in decorator
    kwargs['api_client'] = _get_api_client(config, command_name)
  File "/usr/local/lib/python3.9/site-packages/databricks_cli/configure/config.py", line 104, in _get_api_client
    return ApiClient(host=config.host, token=config.token, verify=verify,
  File "/usr/local/lib/python3.9/site-packages/databricks_cli/sdk/api_client.py", line 106, in __init__
    method_whitelist=set({'POST'}) | set(Retry.DEFAULT_METHOD_WHITELIST),
AttributeError: type object 'Retry' has no attribute 'DEFAULT_METHOD_WHITELIST'
Error: AttributeError: type object 'Retry' has no attribute 'DEFAULT_METHOD_WHITELIST'

Thanks,

Suresh

Re: Databricks fs ls does not work

Suresh_AWS — Sat, 06 May 2023 00:10:16 GMT

I was able to solve the problem by updating the package urllib3 with a different version

pip3 install urllib3=1.25.11

Re: Databricks fs ls does not work

abos — Mon, 08 May 2023 08:04:07 GMT

It seems the Databricks cli needs and update. This issue is discussed on StackOverflow. For the time being, installing an older version of `urllib3` should solve the issue. The method was removed since 2.X, the latest 1.X version is 1.26.15. You can use

pip install urllib3==1.26.15