dbfs rest api - Access denied on dbfs rest api call but access works in dataricks notebook

[PrData05]

Hello, 

We are trying to interact with dbfs to upload files and list files in dbfs directory ( we are tring to upload files in volumes ). Although we have necessary permissions on databricks, we are still getting permission denied when we are making rest api calls.

We are able to work successfully on databricks notebooks with same permissions. 

error code when we are trying to list files in a directory 

{"error_code":"PERMISSION_DENIED","message":"No operations allowed on this path"}

erorr when we are trying to upload files in a directory:

{"error_code":"INVALID_PARAMETER_VALUE","message":"You must provide the upload path!"}

the request to upload file looks like this:

{'path': 'dbfs:/Volumes/sandbox/managed/mvolume/new_file.json', 'contents': 'SGVsbG8sIFdvcmxkIQ=', 'overwrite': 'true'}

Any leads why this is not working, would be appreciated. Thank you 🙂

[Kaniz]

Hi @PrData05 , Could you please Check the upload path, verify that the path parameter in your upload request is correct?

[PrData05]

Hi @Kaniz , I verified , the upload path is correct and I am passing the correct path in the request. 

[TheIceBrick]

@PrData05 did you get anywhere with uploading to a volume? This has been suggested as a workaround for uploading to a volume using the dbfs api, but I am getting the same results.

[PrData05]

the api didn't work for me ,so I used databricks sdk to upload files.

I used WorkspaceClient's upload method. Here's the sample that might help

workspace_client = WorkspaceClient(
        host = SERVER_HOSTNAME,
        token = ACCESS_TOKEN
    )
response = workspace_client.files.upload(file_path, content)

.  

[sgdnp]

I am experiencing the same issue as well. Any ideas about how we can upload to volumes using the api? Tried both the /api/2.0/dbfs/put and the streaming apis (/api/2.0/dbfs/create,/api/2.0/dbfs/add-block,/api/2.0/dbfs/close) but no luck so getting it to work.

[PrData05]

the api didn't work for me ,so I used databricks sdk to upload files.

I used WorkspaceClient's upload method. Here's the sample that might help

workspace_client = WorkspaceClient(
        host = SERVER_HOSTNAME,
        token = ACCESS_TOKEN
    )
response = workspace_client.files.upload(file_path, content)

.