Once PCI-DSS compliance is enabled in Azure Databricks, the workspace is locked into a set of restrictions to maintain those standards and safeguard sensitive data. These restrictions include disabling access to features like serverless compute, which do not currently meet the required PCI-DSS controls for data security and isolation. There is currently no supported way to adjust compliance settings within that workspace to selectively enable serverless compute while keeping compliance active—this is by design, to ensure strict adherence to PCI-DSS requirements.
Managing Feature Gaps with Separate Workspaces
Using a separate, non-compliant Azure Databricks workspace for tasks that require features not allowed under PCI-DSS (such as serverless compute) is a recommended practice from a security, compliance, and operational perspective. This approach is called "workload isolation," and it allows the organization to maintain PCI-DSS compliance for sensitive workflows while utilizing cost-effective or flexible features elsewhere for non-sensitive workloads.
Key strategies for managing multiple workspaces:
-
Clear Data Segregation: Ensure no PCI-sensitive data is moved into or processed in the non-compliant workspace.
-
Access Controls: Enforce strong identity and access management policies, limiting data movement between workspaces to only authorized users or automated, audited processes.
-
Governance and Monitoring: Use centralized tools and logging to retain oversight of workspace usage and data flows. Azure offers features such as Azure Policy, Azure Monitor, and logging integrations for compliance and management across environments.
-
Naming/Tagging Conventions: Consistently label workspaces based on their compliance posture for easy identification and management.
Best Practices and Considerations
-
Always keep strict boundaries between compliant and non-compliant workspaces, including separate compute resources and dedicated storage.
-
Document and regularly review data flows to ensure compliance is not accidentally compromised.
-
Schedule regular audits and reviews of workspace configurations, permissions, and monitoring rules.
-
For any tasks or workloads that require PCI scope or protected data, ensure they only occur in the compliant workspace.
Summary Table
| Feature / Practice |
PCI Workspace |
Non-PCI Workspace |
Notes |
| Serverless Compute |
Not available |
Available |
Use non-PCI workspace for these tasks |
| Data Storage |
PCI-compliant only |
General storage |
No PCI data in non-compliant workspace |
| Access Controls / Monitoring |
Strict, central logging |
Flexible, but monitored |
May use different configuration sets |
Implementing this separation allows organizations to balance compliance and productivity, maintaining PCI standards for sensitive workloads while using advanced Databricks features for general analytics or development.
If additional guidance or architectural support is required, Azure offers reference architectures and compliance documentation for managing these scenarios effectively.
