We follow an "infrastructure as code" model for Workspace deployment, but the limited documentation means that we end up configuring most settings by hand, which is a tedious and time-consuming process. This may be fine for small companies that are only using Databricks to handle non-sensitive data, but our organization is highly regulated and required to enforce NIST 800-53 standards.

It doesn't help that Databricks does not include default configuration maps that implement this compliance, but the absence of configurable settings through an API is honestly inexcusable and hard to justify to our oversight organizations because we cannot run automation to set the values to spec, nor can we run automation to monitor (i.e. get) ongoing compliance of existing workspaces by evaluating their current values.

Databricks must do better if it wants to stay competitive with entrenched cloud competitors' (i.e. AWS and Azure) data management and analysis features. I need Databricks to be better so my teams are not forced to stop using it.