Coarse-grained access management for jobs

[camilo_s]

Are there any perspectives in Databricks' roadmap for enabling coarse-grained access management for jobs?

Currently, access to jobs has to be managed on a job by job basis: https://docs.databricks.com/en/security/auth-authz/access-control/index.html#job-acls . It's not difficult to implement this using the Update job permissions API (e.g. automatically during CI/CD).

However, it is difficult to get a global view on who has access to what, short of collecting that information job-by-job yourself. Now, imagine you have to migrate some service principal and need to update permissions on all the workflows it has access to, what a nightmare.

It would be great if you could define job scopes or folders (like workspace folders) that you could define access permissions on in a hierarchical hereditary way. Having a top level scope could also be a first step towards restricting job creation (e.g. only principals X, Y, Z may create jobs at "root" scope, etc.)

Is it just me lacking such a feature?

[Kaniz_Fatma]

Hi @camilo_s, You’re not alone in your desire for more streamlined access management in Databricks!

Currently, access to jobs is handled on a per-job basis, which can become cumbersome when dealing with multiple workflows. However, there isn’t a built-in feature for defining job scopes or folders with hierarchical access permissions.

That said, it’s a valid request, and I recommend keeping an eye on Databricks’ roadmap. While I don’t have specific information about upcoming features, the platform continually evolves, and user feedback often drives enhancements. In the meantime, you might consider organizing your jobs using naming conventions or other metadata to help manage access more effectively.

If you’d like to explore more about Databricks’ access controls and permissions, you can check out their video guide on workspace access controls. Hopefully, future updates will address your need for a more structured approach to job access management! 

[camilo_s]

Hi @Kaniz_Fatma, thanks for your reply.

A more mature access management concept in Databricks would be definitely terrific. I understand it's not entirely along the AI-lines that Databricks is pushing hard currently, but it would improve a lot the platform experience/platform capabilities (e.g. self-service) of Databricks.

I'm not gonna mark the answer as a solution as the original problem isn't solved. Maybe this discussion serves as a place for gathering feedback around the topic.