Unable to setcheckpointdir in unitycatalog enabled workspace
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-06-2023 06:24 AM - edited 12-06-2023 06:30 AM
I have a Unity catalog enabled workspace where I am trying to setCheckpointDir during runtime. The method looks to authenticate using fs.azure.account.key instead of storage credentials. I am using databricks access connector which has "Storage Blob Data Contributor" role on my storage account. The relevant containers are also added as external locations.
Error:
Py4JJavaError: An error occurred while calling o350.setCheckpointDir.
: Failure to initialize configuration for storage account: Invalid configuration value detected for fs.azure.account.keyInvalid configuration value detected for fs.azure.account.key
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-07-2023 05:46 AM
@Retired_mod I have provided all the necessary permissions and were able to browse through the folders of the container added as an external location.
I don't understand why the method setcheckpointdir looks for account key when the access is already provided using storage credentials for the storage account.
