topic Re: "Azure Container Does Not Exist" when cloning repositories in Azure Databricks in Administration & Architecture

"Azure Container Does Not Exist" when cloning repositories in Azure Databricks

ptco — Tue, 02 Jul 2024 09:54:44 GMT

Good Morning, I need some help with the following issue:

I created a new Azure Databricks resource using the vnet-injection procedure. (here)

I then proceeded to link my Azure Devops account using a personal account token.

If I try to clone a repository I get an error message saying:

The Azure container does not exist

From other occurrences of the same error message in the community I seem to understand that this is something associated with the Unity Catalog, however even when I remove the workspace from the Unity Catalog I get the same error.

Many Thanks

Re: "Azure Container Does Not Exist" when cloning repositories in Azure Databricks

ptco — Wed, 03 Jul 2024 10:36:43 GMT

Hi @Retired_mod many thanks for your reply.

I have verified that repository works in another (older) databricks workspace. When I try to clone into a new databricks instance, it creates subfolders but won't copy the files. I have attached a screencap of the error message for your convenience.

How do I check the Unity Catalog settings for lingering references? The account admin removed this workspace from the Unity Catalog, is there anywhere else where we should be looking at?

Re: "Azure Container Does Not Exist" when cloning repositories in Azure Databricks

orophile_user — Mon, 05 Aug 2024 16:58:32 GMT

Any solution to this, I too facing same issue

Re: "Azure Container Does Not Exist" when cloning repositories in Azure Databricks

NandiniN — Tue, 06 Aug 2024 01:47:58 GMT

There are three possible causes.

The Azure container might not have been properly created when the workspace was provisioned.
The Azure container might have been deleted or moved after it was created.
There might be a problem with the permissions or roles assigned to the user or service principal trying to access the Azure container.

In your case it seems to be the permission issue or if the container is deleted.

There are a couple of potential solutions:

If the workspace is new, you could delete and create a new workspace. This might solve the issue if the problem was caused by an error during the provisioning process.
You could also reach out to the PG team to investigate what happened during the provisioning of the workspace and why the Azure container was not created.

Re: "Azure Container Does Not Exist" when cloning repositories in Azure Databricks

Dnirmania — Tue, 17 Dec 2024 12:55:20 GMT

I've come across similar responses in many articles regarding the "Azure Container Does Not Exist" error, but none of them have been helpful. In both the error and responses, people suggest checking the Azure container, but no one specifies where to check or what permissions are needed. If anyone could provide detailed troubleshooting steps for this error, it would be greatly appreciated.

Re: "Azure Container Does Not Exist" when cloning repositories in Azure Databricks

kieranswhite — Tue, 17 Dec 2024 19:11:30 GMT

I found my error was related to my use of Pulumi (or in this other answer Terraform)

Re: "Azure Container Does Not Exist" when cloning repositories in Azure Databricks

NandiniN — Wed, 08 Jan 2025 23:23:08 GMT

@Dnirmania

To troubleshoot the "Azure Container Does Not Exist" error, follow these steps:

Confirm the Azure Container Exists:
- Verify that the Azure container exists in the specified resource group and subscription. If it has been deleted or moved, recreate it in the correct location.
Check Permissions:
- Ensure that the user or service principal has the necessary permissions to access and create files in the Azure container. Adjust the roles and permissions as needed.
Workspace Provisioning:
- If the workspace is newly deployed and the issue persists, consider deleting and recreating the workspace. This can resolve issues caused by errors during the initial provisioning process.
Investigate Provisioning Errors:
- If the Azure container was not properly created during the workspace provisioning, there might have been an error during the process. You can reach out to the product team to investigate what happened during the provisioning of the workspace and why the Azure container was not created.

By following these steps, you should be able to resolve the "Azure Container Does Not Exist" error.

Re: "Azure Container Does Not Exist" when cloning repositories in Azure Databricks

NandiniN — Wed, 08 Jan 2025 23:32:39 GMT

@Dnirmania

Please find this KB I just landed on - https://kb.databricks.com/en_US/cloud/azure-container-does-not-exist-error-when-creating-files-in-a-workspace

Re: "Azure Container Does Not Exist" when cloning repositories in Azure Databricks

ptco — Thu, 09 Jan 2025 09:51:52 GMT

Hello,

my understanding is that some unspecified error happens in the managed resource group that DB creates when you deploy a new instance. Depending on your admin right you might not even be able to see such group in Azure (as in my case).

What has worked for me was to delete the instance and redeploy from scratch. I have deployed several new instances of DB in different resource groups since my original post and the problem has occasionally surfaced again, but deleting and redeploying has always done the trick.

Hope it helps!

Re: "Azure Container Does Not Exist" when cloning repositories in Azure Databricks

Dnirmania — Fri, 10 Jan 2025 16:01:29 GMT

Thank you Everyone for helping me with this issue.. My issue got resolved after recreating the workspace.

Re: "Azure Container Does Not Exist" when cloning repositories in Azure Databricks

ihenadb — Tue, 21 Jan 2025 12:55:59 GMT

Also having problems with this after some IaC testing deleting and recreating the workspace with the same name. We are working in Azure.

Is the "container" referred to, the container in the storage account deployed by Databricks Instance into the managed resource group ?

Thanks

Re: "Azure Container Does Not Exist" when cloning repositories in Azure Databricks

NandiniN — Sun, 26 Jan 2025 07:29:00 GMT

Yes, the container is indeed the container in the storage account deployed by the Databricks instance into the managed resource group. This storage account is part of the managed resource group associated with your Azure Databricks workspace.

If the workspace is newly deployed workspace, consider deleting and recreating the workspace.

Can you please try recreating the