topic Is there a setting which restricts users from Creating Job and Pipeline? in Administration & Architecture

Is there a setting which restricts users from Creating Job and Pipeline?

Avvar2022 — Mon, 25 Mar 2024 18:49:57 GMT

as far i know currently ((as of 03-25-2024) databricks don't any workspace admin settings option to restrict users from creating a workflow/job or delta pipelines.

Here is the use case for it Example: you have 3 tier landscape Dev, Qa and Prod.

It is ok for users to create workflows/jobs and pipelines in dev but there is no need for them to create in QA and production but as an admin we can't restrict users from creating.

If anyone else out there feels same. Unbale to restrict could have below disadvantages.

1. There could lot unnecessary jobs and if user base large there could be many workflows.

2. Let's assume user has read access to data and access to all purpose cluster/SQL warehouse, user can schedule a workflow. as admin we expect user not to do it but admin won't be able to restrict. This could incur cost.

There could workarounds like scheduling a job to delete workflows which are created by users, but it would be good to have this feature.

Re: Is there a setting which restricts users from Creating Job and Pipeline?

Avvar2022 — Fri, 07 Jun 2024 19:41:00 GMT

@Retired_mod - How can I initiate a product enhancement request?

Re: Is there a setting which restricts users from Creating Job and Pipeline?

camilo_s — Tue, 18 Jun 2024 08:23:44 GMT

We face the same situation as the OP and are likewise missing such a feature. In my opinion this limits Databricks' platform capabilities (speaking from the perspective of a platform team that intends to provide Databricks workspaces as a service to internal consumers).

A great solution would be the possibility to define workspace-wide policies, and one example directed to the OPs and my situation could be defining a policy that restrict the possible owners of workflows, as in "Only this or that principal (user/ group/ service principal) may create/own workflows". While we're at that, why not also a policy to restrict the possible "run as" principals for a workflow?

Re: Is there a setting which restricts users from Creating Job and Pipeline?

camilo_s — Tue, 18 Jun 2024 08:29:38 GMT

I notice that a separate discussion overlaps with the OPs issue: https://community.databricks.com/t5/data-engineering/restricting-workflow-creation-and-implementing-approval/td-p/4336

@Retired_mod do you have a mechanism for clustering discussions for contributors to have a better view of existing topics and Databricks to know which major topics its customers occupy with?

I'm thinking about issue tracking in GitHub, where moderators often track topics in a single GitHub issue, closing related ones and referring them to the single issue.