Dedicated Access Mode Interactive Cluster with a Service Principal

[73334]

Hi, I am wondering if it is possible to set up an interactive cluster set to dedicated access mode and having that user be a machine user?

I've tried the cluster creation API, /api/2.1/clusters/create, and set the user name to the service principal name but that did not work, stating the user could not be found. Trying it via the cluster creation UI also does not work since the user selection dropdown does not show machine users. 

It seems like its possible to create dedicated access mode clusters for workflows but not interactive clusters.

Is there any way to do this, if not will it ever be supported? There seems to be a bit of documentation stating support for having machine users set as the user for dedicated clusters.

[Isi]

Hello @73334 ,

I tested and its possible.  You have to use the application-id.

Hope this helps, 🙂

Isi

[Coffee77]

Not only is that possible but totally recommendable for automation purposes. You can create or update those clusters via Databricks CLI with cluster properties configured with proper JSON file. Take a look to the sample code I am using to update a cluster with a service principal account. Notice reference to service principal ID in single_user_name property:

Automation Cluster with Service Principal

By the way, I'm about to publish a video in my youtube channel about deployment with DABs plus other resources like this, not already included as far as I know. 

[Coffee77]

It turns out that now is possible to include deployment of interactive and SQL Warehouse clusters with Databricks Asset Bundles, so you can include a YAML file similar to this one to deploy that type of interactive clusters:

Definition of Interactive Dedicated cluster with Service Principal:

DAB configuration file: