Error: the Service Account Key in storage credential is not configured correctly

Data Engineering

Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.

We have databricks on GCP. Streamings are running 24/7, storage credentials and external location are created as we are using managed unity catalog. We get random error, somewhere are around mid night (UTC). Here is trace of Error,

ERROR MicroBatchExecution: [streaming] terminated with error
java.util.concurrent.ExecutionException: com.databricks.sql.managedcatalog.acl.UnauthorizedAccessException: PERMISSION_DENIED:
Failed to get credentials: the Service Account Key in storage credential is not
configured correctly. Please contact your account admin to update the configuration.