Error: the Service Account Key in storage credential is not configured correctly

[Sadam97]

We have databricks on GCP. Streamings are running 24/7, storage credentials and external location are created as we are using managed unity catalog. We get random error, somewhere are around mid night (UTC). Here is trace of Error,

ERROR MicroBatchExecution: [streaming] terminated with error
java.util.concurrent.ExecutionException: com.databricks.sql.managedcatalog.acl.UnauthorizedAccessException: PERMISSION_DENIED:
Failed to get credentials: the Service Account Key in storage credential is not
configured correctly. Please contact your account admin to update the configuration.

 

[Kaniz_Fatma]

Hi @Sadam97, 

• Could you please ensure that the Service Account Key associated with your Databricks workspace has the necessary permissions to access the storage resources (such as Google Cloud Storage or other external storage)? Please verify that the key is correctly configured in your Databricks workspace.
• Since you mentioned using a managed Unity catalog, ensure that the catalog is set up correctly. Please confirm that the external location and storage credentials are properly configured within the Unity catalog.
• If the errors occur around midnight (UTC), consider checking any scheduled jobs or processes that run during that time.

When seeking further assistance, remember to provide additional context or logs if available. If you need more specific guidance, feel free to ask!