"Azure Container Does Not Exist" when cloning repositories in Azure Databricks

[ptco]

Good Morning, I need some help with the following issue:

I created a new Azure Databricks resource using the vnet-injection procedure. (here)

I then proceeded to link my Azure Devops account using a personal account token. 

If I try to clone a repository I get an error message saying:

The Azure container does not exist

From other occurrences of the same error message in the community I seem to understand that this is something associated with the Unity Catalog, however even when I remove the workspace from the Unity Catalog I get the same error.

Many Thanks

[ptco]

Hi @Retired_mod many thanks for your reply.

I have verified that repository works in another (older) databricks workspace. When I try to clone into a new databricks instance, it creates subfolders but won't copy the files. I have attached a screencap of the error message for your convenience.

How do I check the Unity Catalog settings for lingering references? The account admin removed this workspace from the Unity Catalog, is there anywhere else where we should be looking at?

[orophile_user]

Any solution to this, I too facing same issue

[NandiniN]

There are three possible causes. 

• The Azure container might not have been properly created when the workspace was provisioned.
  
• The Azure container might have been deleted or moved after it was created.
  
• There might be a problem with the permissions or roles assigned to the user or service principal trying to access the Azure container.

In your case it seems to be the permission issue or if the container is deleted. 

There are a couple of potential solutions:

• If the workspace is new, you could delete and create a new workspace. This might solve the issue if the problem was caused by an error during the provisioning process.
• You could also reach out to the PG team to investigate what happened during the provisioning of the workspace and why the Azure container was not created.

[Dnirmania]

I've come across similar responses in many articles regarding the "Azure Container Does Not Exist" error, but none of them have been helpful. In both the error and responses, people suggest checking the Azure container, but no one specifies where to check or what permissions are needed. If anyone could provide detailed troubleshooting steps for this error, it would be greatly appreciated.

[ptco]

Hello,

my understanding is that some unspecified error happens in the managed resource group that DB creates when you deploy a new instance. Depending on your admin right you might not even be able to see such group in Azure (as in my case).

What has worked for me was to delete the instance and redeploy from scratch. I have deployed several new instances of DB in different resource groups since my original post and the problem has occasionally surfaced again, but deleting and redeploying has always done the trick.

Hope it helps!

[kieranswhite]

I found my error was related to my use of Pulumi (or in this other answer Terraform)

[NandiniN]

@Dnirmania 

To troubleshoot the "Azure Container Does Not Exist" error, follow these steps:

1. Confirm the Azure Container Exists:

   • Verify that the Azure container exists in the specified resource group and subscription. If it has been deleted or moved, recreate it in the correct location.

2. Check Permissions:

   • Ensure that the user or service principal has the necessary permissions to access and create files in the Azure container. Adjust the roles and permissions as needed.

3. Workspace Provisioning:

   • If the workspace is newly deployed and the issue persists, consider deleting and recreating the workspace. This can resolve issues caused by errors during the initial provisioning process.

4. Investigate Provisioning Errors:

   • If the Azure container was not properly created during the workspace provisioning, there might have been an error during the process. You can reach out to the product team to investigate what happened during the provisioning of the workspace and why the Azure container was not created.

By following these steps, you should be able to resolve the "Azure Container Does Not Exist" error.

[NandiniN]

@Dnirmania  

Please find this KB I just landed on - https://kb.databricks.com/en_US/cloud/azure-container-does-not-exist-error-when-creating-files-in-a-...

[Dnirmania]

Thank you Everyone for helping me with this issue.. My issue got resolved after recreating the workspace.