Control Databricks Platform version

[juanjomendez96]

Hello there! ✌️

I have noticed that my Databricks UI has been changed from time to time, I was wondering how can I control the Databricks Platform version so I don't keep having new changes and new ways/names in my UI. 

I have found a release page https://docs.databricks.com/aws/en/release-notes where they explain what it is included in each release. Although, I have not found any documentation for how to control this changes and versions.

From my point of view, we should be able to control what is changed and when it is changed, so I was wondering if someone from this community knows how to control it.

PS. we are using Terraform to control the deployment of our Databricks Platform, so maybe we need to control it from there ¿?

Thanks in advance!

[Khaja_Zaffer]

Hello @juanjomendez96 

Updates to the control plane and user interface are typically automatically rolled out in staged deployments across regions and workspaces because Databricks manages the core platform, including the user interface, as a SaaS service.

This means that, unlike self-hosted software, you cannot directly pin or control the version of the entire platform. 
As you can see from the release notes, there is no built-in way to opt out of platform UI general availability (GA) updates.

Manage/resistrick your databricks account 

As a workspace or account admin, go to the Previews page in your workspace (accessible via the admin settings menu under your username in the top bar). More details on this can be found on the website : 

https://docs.databricks.com/aws/en/admin/workspace-settings/manage-previews#manage-previews 

If you are trying to toggle between databricks sql current to preview or vice versa 

If you want to switch between versions (Current → Preview or vice versa), you need to restart the SQL Warehouse. This works similarly to Databricks Runtime, but if the warehouse is set to Current, it will automatically update when Databricks releases a new stable version

Now, as you mentioned about terraform being used to control the deployment, may be see if there is any code which has databricks_runtime_version is showing preview rather than current.

I am also open for other solutions for this case. 

[juanjomendez96]

Hello @Khaja_Zaffer !

thanks for the fast response, really appreciate it.

It is a pity that we do not have such control over what is deployed in our control plane, but I'm sure Databricks has a heavy reason to do this way.

Anyhow, I will talk with our Devops team to see if there is such option in the Terraform code, but AFAIK, there is not any way to control it from it (in the past they told me that there were not such option, maybe it has changed, who knows...)

[Khaja_Zaffer]

Hello @juanjomendez96 

Do you have any specific example where you dont want to see the changes and you feel like this changes are not required on your workspace?

[juanjomendez96]

well, for example, it seems the latest version of the Databricks Control Plane does not work properly for the apps, now I cannot assign any new permissions or revoke the existing ones. One week ago it worked like a charm, so I assume is due to the latest update in the Control Plane, that is why I would like to control the version used for the control plane...

[Khaja_Zaffer]

Can you please share any screenshot that can help us. 

There must be a workaround for sure. Because the updates don't over kill such things 

[juanjomendez96]

unluckily I cannot share any screenshot, but the problem is as follows:

We click in the 'Permissions' button.

Then we add the groups to grant access.

Click on 'Add'.

Click on 'Save' button.

After this, we go again to the permissions section and the groups granted previously are not there. 

Not sure if this has anything to do with the control plane version, but it seems that databricks team is changing a lot of things and we cannot track every new feature changed from the past. Another example could be the naming they follow, you learn something as it is right now and nothing confirms you that it will never change unexpectedly, we should be able to control the version of the control plane to avoid this kind of situations...

[Khaja_Zaffer]

I understand your frustaton @juanjomendez96  but thats how the data world these days very much changing everyday to meet demands. 

Anyhow, https://learn.microsoft.com/en-gb/azure/databricks/dev-tools/databricks-apps/permissions is the way to change the things which you are looking for. 

If you are unable to make the changes by UI 

You have alternative way!

Open a SQL Editor or Notebook in the workspace.

Run:

sql

-- Grant to a group (replace placeholders)
GRANT CAN_USE ON APP '<app-name-or-id>' TO `group-name`;

-- to Verify
SHOW GRANTS ON APP '<app-name-or-id>';

-- for revoke
REVOKE CAN_USE ON APP '<app-name-or-id>' FROM `group-name`;

This directly hits the backend and should persist if the UI is the culprit. Apps support group-based grants like users.

I hope this works. 

[juanjomendez96]

Thanks for the response @Khaja_Zaffer , I have tried it but I cannot make it work. It keeps telling me that I have a syntax error in the SQL command (my application is called 'django-poc' and the group 'data_scientists')

GRANT CAN_USE ON APP 'django-poc' TO `data_scientists`;

And the error I receive is: Syntax error at or near ''django-poc''. SQLSTATE: 42601

Not sure what I'm doing wrong to be honest, do you have any clue by any chance?

Thanks in advance for your support, you have been really helpful.

[juanjomendez96]

@Khaja_Zaffer reviewing the documentation it seems the type APP does not exists yet (see options in https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-privileges#securable-objects).

I will wait for them to fix the permissions UI issue or for them to add the type APP to the GRANT command. 

Thanks anyways for your efforts.

[Khaja_Zaffer]

Yeah, we can only do it by UI/API for apps.

I thought it would work via sql just like we do it for tables/schema. 

[Khaja_Zaffer]

I think to enable a app via sql there is some limitation. I am sorry for that. 

Do you have any idea enabiling via API?

[Khaja_Zaffer]

Hello @juanjomendez96 (edited)

yes indeed you are right. There were some updates 

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-apps/permissions 

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-apps/auth

[Khaja_Zaffer]

As per the document which is new: 

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-apps/auth#app-authorization 

GRANT CAN USE ON APP `django-poc` TO `data_scientists`; should have worked! not sure what it is.