07-28-2023 06:44 AM
I want to upload a simple csv file to a volume which was created in our unity catalog. We are using secure cluster connectivity and our storage account (metastore) is not publicly accessable. We injected the storage in our vnet.
I am getting the following error while uploading:
"An error occurred while uploading. Please try again"
Should i open the IP Address of the control plane in our storage account? Is there a way to fix this?
08-01-2023 06:29 AM
Hi @Martinitus
I believe this might be related to one of the known limitations listed in our documentation:
"You cannot upload or download files for volumes backed by Azure storage accounts configured with Azure Firewall or Private Link" Source
We are actively working on this and hope for a fix in the coming weeks.
07-28-2023 08:49 AM
@boriste can you please check if you have permissions to read/write to volume, or you admin if not please reach out to admin to provide to access to write/read to your volume
07-30-2023 11:10 PM - edited 07-30-2023 11:11 PM
@karthik_p I have full admin rights and im logged in with my azure ad account. I also have read/write permissions
But im getting the message:
{ "error_code" : "BAD_REQUEST", "message" : "Missing credentials to access the DBFS root storage container in Azure." }
I dont know if databricks uses another identity but somehow i cant upload
07-31-2023 01:43 AM
We (boriste and me) have tested creating volumes from a notebook which worked just fine. This means, networking and auth from the cluster (dataplane) to the metastore storage account works.
What was not working was uploading a file via the Web-UI. Our suspicion is, that this upload tries to save the file to the metastore storage account from the databricks control plane. Which does _not_ have access (networking is blocked by NSG) to our metastore storage account.
I assume one solution would be to whitelist the IP of the databricks controlplane in the networking settings of our metastore storage account. We could try that but we don't know the appropriate IP range.
07-31-2023 02:15 AM
We checked the storage account logs of the metastore. Whenever we try to upload a file to a volume from the UI of the workspace, we get an authorization error. The request seems to be coming from a Databricks Java Backend service (ControlPlane?). We tried adding the caller IP to the storage account networking white list - that is not possible because its not a public IP?
08-01-2023 06:29 AM
Hi @Martinitus
I believe this might be related to one of the known limitations listed in our documentation:
"You cannot upload or download files for volumes backed by Azure storage accounts configured with Azure Firewall or Private Link" Source
We are actively working on this and hope for a fix in the coming weeks.
10-12-2023 03:57 AM
Hello! Do you have an idea as to when this will be fixed - what release is it aligned to? As this is still an issue and is causing me and my customers some issues, considering that you can't use DBFS with Unity Catalog.
10-24-2023 07:13 AM
Hi @UstOldfield
A fix was released for the previous issues where upload/download was not working for Azure storage accounts configured with Azure Firewall or Private Link.
Can you try to see if this addresses your problem?
10-24-2023 07:32 AM
Hey @AdrianaIspas - it works. Thanks for letting me know.
10-24-2023 03:21 PM
A fix was released for the previous issues where upload/download was not working for Azure storage accounts configured with Azure Firewall or Private Link.
10-09-2023 04:38 AM
@AdrianaIspas We are running into the same issue. It took a while to figure out that the error message is related to this limitation. Any updates on when we can expect the limitation to be taken away? We want to secure access to our storage accounts with a firewall, setting up a public storage account is not an option.
10-24-2023 03:20 PM
Hi @jeroenvs
A fix was released for the previous issue where upload/download was not working for Azure storage accounts configured with Azure Firewall or Private Link.
Can you try to see if this addresses your problem?
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group