Hi @colinsorensen, Based on the given information, it appears that the error you are encountering is related to accessing cloud storage in Databricks. The error message suggests that the cluster does not have the necessary permissions to access the storage.
To troubleshoot this issue, you can follow these steps:
1. Check the driver log for complete error details. The error message may provide more specific information about the cause of the issue.
2. If the error includes a traceId, you can search for it in Kibana if you have ELK access. This can help you find additional error details related to the issue.
3. Collect the storage account details and storage credentials associated with the cluster. Ensure that the cluster's storage principal or managed identity has the necessary permissions to access the storage account. Specifically, it should have the STORAGE BLOB DATA CONTRIBUTOR role assigned at the storage account level.
4. Verify that no storage firewall issues could prevent the cluster from accessing the storage.
5. Double-check the storage credentials or Spark Azure keys the cluster uses.
Ensure that they are correct and valid. It's important to note that this error is not necessarily a bug in Databricks itself but rather an issue with the configuration or permissions related to accessing cloud storage.
