Just some background. I deliver quite some trainings on Azure Databricks and in these trainings students have to provisions their own Azure Databricks workspace and UC. In the past this always went smoothly.
Go to the Azure Portal
- Create a storage account
- Create workspace
- Grant Storage Blob Contributor on the storage account to an Access Connector of Azure Databricks
- Create a new UC metastore and link the workspace to it.
- Done.
This whole procedure did actually create the external location and connection objects automatically.
When we now try to repeat these same 5 steps we get an error creating the UC metastore:
parent external location for path `abfss://<>@<>.dfs.core.windows.net/` does not exist.
Some other strange thing is that the abfss:// never had to be added in the past. Now you must manually add it. In the docs it is still saying "The abfss:// prefix is added automatically."
To get the creation of the of the UC metastore to succeed, the Credential and External Location must be created upfront. Once that is done, the above error message does not appear anymore...
Any idea what is causing the changed behaviour?
