10-19-2023 11:12 AM
Hello,
We are trying to interact with dbfs to upload files and list files in dbfs directory ( we are tring to upload files in volumes ). Although we have necessary permissions on databricks, we are still getting permission denied when we are making rest api calls.
We are able to work successfully on databricks notebooks with same permissions.
error code when we are trying to list files in a directory
{"error_code":"PERMISSION_DENIED","message":"No operations allowed on this path"}
erorr when we are trying to upload files in a directory:
{"error_code":"INVALID_PARAMETER_VALUE","message":"You must provide the upload path!"}
the request to upload file looks like this:
{'path': 'dbfs:/Volumes/sandbox/managed/mvolume/new_file.json', 'contents': 'SGVsbG8sIFdvcmxkIQ=', 'overwrite': 'true'}
Any leads why this is not working, would be appreciated. Thank you 🙂
10-20-2023 10:57 AM
Hi @Retired_mod , I verified , the upload path is correct and I am passing the correct path in the request.
01-22-2024 03:08 AM
@PrData05 did you get anywhere with uploading to a volume? This has been suggested as a workaround for uploading to a volume using the dbfs api, but I am getting the same results.
02-16-2024 11:47 AM
the api didn't work for me ,so I used databricks sdk to upload files.
I used WorkspaceClient's upload method. Here's the sample that might help
workspace_client = WorkspaceClient(
host = SERVER_HOSTNAME,
token = ACCESS_TOKEN
)
response = workspace_client.files.upload(file_path, content)
.
02-16-2024 11:40 AM
I am experiencing the same issue as well. Any ideas about how we can upload to volumes using the api? Tried both the /api/2.0/dbfs/put and the streaming apis (/api/2.0/dbfs/create,/api/2.0/dbfs/add-block,/api/2.0/dbfs/close) but no luck so getting it to work.
02-16-2024 11:48 AM
the api didn't work for me ,so I used databricks sdk to upload files.
I used WorkspaceClient's upload method. Here's the sample that might help
workspace_client = WorkspaceClient(
host = SERVER_HOSTNAME,
token = ACCESS_TOKEN
)
response = workspace_client.files.upload(file_path, content)
.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group