Re: Error trying to edit Job Cluster via Databrick...

Louis_Frolio · ‎10-28-2025

Hey @Adam_Borlase , Thanks for sharing the command and error—this is a common pitfall when trying to control Lakeflow (DLT) compute with cluster policies.

What the error means

The message “NO_ISOLATION or custom access modes are not allowed in this workspace” indicates your workspace has been configured to disallow the legacy “No isolation shared” and “Custom” access modes. Admins can hide or block the “No isolation shared” mode in the workspace settings; when a cluster or policy attempts to use it, you’ll get exactly this error. Databricks recommends using modern access modes (Standard or Dedicated) rather than “No isolation shared,” which is considered legacy and not recommended.

Why editing the cluster won’t apply to DLT/Lakeflow pipelines

Lakeflow Declarative Pipelines (formerly DLT) create ephemeral pipeline clusters on-the-fly. You don’t edit those clusters directly; instead, you attach a compute policy to the pipeline’s compute settings (default and maintenance clusters) so that Databricks enforces the policy when it provisions the pipeline’s compute. Policies for pipeline compute should be written with cluster_type set to dlt, and you can have policy defaults automatically applied by setting apply_policy_default_values to true in the pipeline config.

How to apply a compute policy to Lakeflow (DLT) compute

Use one of these supported paths:

UI: Open the pipeline, click Settings, uncheck Serverless if you want classic compute, then select your compute policy in the Compute section and Save. This attaches the policy to both the update and maintenance clusters by default.
API/CLI (recommended for reproducibility): Update the pipeline definition to include the policy on the clusters definition and apply policy defaults, for example: json { "clusters": [ { "label": "default", "policy_id": "<policy-id>", "apply_policy_default_values": true } ] } In the policy itself, include "cluster_type": { "type": "fixed", "value": "dlt" } so it is selectable for pipelines.

Important: Do not set autotermination_minutes in policies for pipeline compute—the pipeline shuts down its own compute, and this policy setting will cause an error.

Fix your policy’s access mode

Your policy (or cluster spec) is likely setting either NO_ISOLATION (legacy “No isolation shared”) or trying to use a legacy/custom mode. In a Unity Catalog-enabled workspace, use one of the UC-compatible access modes:

Use data_security_mode = USER_ISOLATION to get Standard access mode (multi-user, isolated).
Or use data_security_mode = SINGLE_USER to get Dedicated access mode (assigned to one principal—user or group).

Also avoid legacy confs like spark_conf.spark.databricks.cluster.profile in policies; forbid them if necessary:

json
"spark_conf.spark.databricks.cluster.profile": {
  "type": "forbidden",
  "hidden": true
}

If you truly need “No isolation shared”

Workspace admins can allow or hide “No isolation shared” from Admin settings. If your organization has enforced user isolation, “No isolation shared” is blocked and any attempt to use it will fail as you observed. Given current best practices, Databricks recommends staying on Standard or Dedicated access modes rather than enabling “No isolation shared.”

What’s wrong with the CLI command A few issues:

clusters edit modifies an existing interactive cluster, not pipeline-created compute. Editing the ephemeral pipeline cluster will not stick; attach the policy to the pipeline instead (UI or API as above).
The token “clusterid 16.4.x-scala2.13” is not a cluster ID—it looks like a runtime version string. The clusters edit command expects an actual cluster ID; pipeline clusters are managed by the pipeline and should be controlled via the pipeline config rather than clusters edit.
To get the desired effect, use the pipeline update path and include policy_id and apply_policy_default_values in the clusters definition; ensure your policy includes cluster_type = dlt and a UC-compatible data_security_mode.

Hope this helps, Louis.