cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Restrictions on setting environment variables in Compute Policies

tcmx
New Contributor II

As recommended by Databricks, we are trying to use Compute Policies to set environment variables, which are used by our notebooks, across clusters.

However, when specifying a JSON string as env var, we are getting this error upon applying the policy to a shared cluster: 

Cannot update compute configuration:
$$, \, ", ', | and backticks are not allowed in the environment variable value for Unity Catalog shared clusters. Please use single user clusters instead.
 
Where is this error coming from? It's not documented anywhere, and seems arbitrary. 
What are possible ways around this?  
 
1 ACCEPTED SOLUTION

Accepted Solutions

-werners-
Esteemed Contributor III

This is because you use Shared access mode.
This enables multiple users to use the cluster simultaneously.
However, there are features that do not work on these Shared access mode clusters:
https://docs.databricks.com/en/compute/access-mode-limitations.html#shared-access-mode-limitations-o...
The limitation about backticks in env vars is not mentioned but the error message is clear: it is not supported.
To overcome this limitation, use Single user as access mode.

View solution in original post

1 REPLY 1

-werners-
Esteemed Contributor III

This is because you use Shared access mode.
This enables multiple users to use the cluster simultaneously.
However, there are features that do not work on these Shared access mode clusters:
https://docs.databricks.com/en/compute/access-mode-limitations.html#shared-access-mode-limitations-o...
The limitation about backticks in env vars is not mentioned but the error message is clear: it is not supported.
To overcome this limitation, use Single user as access mode.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group