topic Re: dbfs rest api - Access denied on dbfs rest api call but access works in dataricks notebook in Data Engineering

dbfs rest api - Access denied on dbfs rest api call but access works in dataricks notebook

PrData05 — Thu, 19 Oct 2023 18:12:08 GMT

Hello,

We are trying to interact with dbfs to upload files and list files in dbfs directory ( we are tring to upload files in volumes ). Although we have necessary permissions on databricks, we are still getting permission denied when we are making rest api calls.

We are able to work successfully on databricks notebooks with same permissions.

error code when we are trying to list files in a directory

{"error_code":"PERMISSION_DENIED","message":"No operations allowed on this path"}

erorr when we are trying to upload files in a directory:

{"error_code":"INVALID_PARAMETER_VALUE","message":"You must provide the upload path!"}

the request to upload file looks like this:

{'path': 'dbfs:/Volumes/sandbox/managed/mvolume/new_file.json', 'contents': 'SGVsbG8sIFdvcmxkIQ=', 'overwrite': 'true'}

Any leads why this is not working, would be appreciated. Thank you 🙂

Re: dbfs rest api - Access denied on dbfs rest api call but access works in dataricks notebook

PrData05 — Fri, 20 Oct 2023 17:57:34 GMT

Hi @Retired_mod , I verified , the upload path is correct and I am passing the correct path in the request.

Re: dbfs rest api - Access denied on dbfs rest api call but access works in dataricks notebook

TheIceBrick — Mon, 22 Jan 2024 11:08:20 GMT

@PrData05 did you get anywhere with uploading to a volume? This has been suggested as a workaround for uploading to a volume using the dbfs api, but I am getting the same results.

Re: dbfs rest api - Access denied on dbfs rest api call but access works in dataricks notebook

sgdnp — Fri, 16 Feb 2024 19:40:39 GMT

I am experiencing the same issue as well. Any ideas about how we can upload to volumes using the api? Tried both the /api/2.0/dbfs/put and the streaming apis (/api/2.0/dbfs/create,/api/2.0/dbfs/add-block,/api/2.0/dbfs/close) but no luck so getting it to work.

Re: dbfs rest api - Access denied on dbfs rest api call but access works in dataricks notebook

PrData05 — Fri, 16 Feb 2024 19:47:32 GMT

the api didn't work for me ,so I used databricks sdk to upload files.

I used WorkspaceClient's upload method. Here's the sample that might help

workspace_client = WorkspaceClient( host = SERVER_HOSTNAME, token = ACCESS_TOKEN ) response = workspace_client.files.upload(file_path, content)

Re: dbfs rest api - Access denied on dbfs rest api call but access works in dataricks notebook

PrData05 — Fri, 16 Feb 2024 19:48:09 GMT

the api didn't work for me ,so I used databricks sdk to upload files.

I used WorkspaceClient's upload method. Here's the sample that might help

workspace_client = WorkspaceClient( host = SERVER_HOSTNAME, token = ACCESS_TOKEN ) response = workspace_client.files.upload(file_path, content)