Store a secret only accessible to the current user

[dvmentalmadess]

During an interactive notebook session, I want a user to be able to retrieve a secret specific to that user. I haven't decided on storage mechanisms, but I'm open to storage mechanisms that can scalably authorize access to a single user and that I can write the secret from an external service. I have looked into the following:

• Databricks Secrets: with a limit of 100 scopes, this does not scale beyond 100 users and I work in an engineering organization with over 200 people
• IAM credential passthrough: does not support MLFlow (my data science team uses MLFlow), and according to my reading it does not support non-admin users calling Scala (I have at least one team that requires the use of Scala)
• Table Access Control: I could use this to create a view that is limited to results matching CURRENT_USER, but won't work for users who need to use Scala
• Workspace object access control: it has an API I can use to write secrets, and I can limit access by user. I would prefer if I can prevent admins from reading the secret of another user, but I haven't figured out if this is possible yet.

I'm thinking workspace object access control is a good option. Can anyone tell me if admin users automatically have access to all objects in a workspace? Is there anything I may have missed that would compromise this solution? Are any of my assumptions incorrect? Are there viable alternatives I'm missing?

[Kaniz_Fatma]

Hi @Mark Miller​ ,

By default, all users can create and modify workspace objects—including folders, notebooks, experiments, and models—unless an administrator enables workspace access control.

With workspace access control, individual permissions determine a user’s abilities.

This article describes how to enable workspace access control and prevent users from seeing workspace objects they do not have access to.

For information about assigning permissions and configuring workspace object access control, see Workspace object access control.

[Kaniz_Fatma]

Hi @Mark Miller​ , Just a friendly follow-up. Do you still need help, or does my response help you to find the solution? Please let us know.

[dvmentalmadess · Accepted solution]

I ended up using Databricks Secrets as the storage mechanism after learning from my account rep that the limit is soft and we can request a higher scope limit. In this case, each user gets a dedicated scope and no other users have access.