Need Guidance on Key Rotation Process for Storage Customer-Managed Keys in Databricks Workspace
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-27-2024 07:19 PM
Problem Statement: We are currently utilizing customer-managed keys for Databricks compute encryption at the workspace level. As part of our key rotation strategy, we find ourselves needing to bring down the entire compute/clusters to update storage encryption keys. However, we encounter errors when attempting to update storage encryption keys without shutting down the compute.
Our workspace is shared by multiple application teams, each with automated jobs triggering compute/clusters to start. The process of stopping all workflows/jobs manually is time-consuming. Is there a way to temporarily pass access at workspace level and allow only Databricks admins to facilitate this key rotation process
Any guidance or best practices on handling key rotations in a shared workspace environment would be greatly appreciated.
Thank you.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-27-2024 07:57 PM
Maybe you can use azure key vault to store customer-managed keys
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-27-2024 08:02 PM
@feiyun0112 Thanks for the reply. the question is how do I stop access temporarily to Databricks workspace for all users except Databricks ADMIN AD group? our workspaces sync with Azure EntraID via SCIM.