cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

Managing Spark Declarative Pipelines Permissions

nulltype
New Contributor II

We are currently managing our permissions via Terraform (including cluster creation, UC governance, etc.). We have a specific `data_engineer` role, and we need everyone with this role to be able to view and manage all of our SDPs.

The Issue: Currently, only Workspace Admins can view and manage these pipelines. The only workaround I have found is to define every individual pipeline within Terraform to manage the ACLs, but this adds significant overhead.

As a small team, we simply need our Data Engineers to have visibility and control over all SDPs without making them full Admins. Does anyone know of a cleaner solution or a group-level permission setting that would solve this?

1 REPLY 1

Abeshek
New Contributor

This is a common governance challenge when teams move to Terraform-driven Databricks environments, trying to balance least-privilege access with operational efficiency. We see this frequently as platforms scale beyond a small admin group.

At Kanerika, as a Databricks partner, we help teams design role-based governance models that give data engineers the right level of visibility and control without expanding workspace admin access or increasing operational overhead.

If you’re open to it, happy to walk through how we typically approach this. Let me know a convenient date and time.

Regards,
Abeshek CH
abeshek.cherukuri@kanerika.com