cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Run Delta Live Tables as service principal

knutasm
New Contributor III

How to run a delta live tables pipeline in production? It uses the owner's (creator's) permissions for writing to tables, and I can't change the owner of a UC-enabled pipeline after creation. I don't want regular users to have write access to prod tables, but run as a service principal instead.

I can find no example in the documentation, but it seems like such a glaring shortcoming that I am sure I must have missed something.

6 REPLIES 6

Kaniz_Fatma
Community Manager
Community Manager

Hi @knutasmTo run a Delta Live Tables pipeline in production with a service principal, ensure the service principal is the pipeline's owner. As per the provided information, the pipeline owner is the one whose permissions are used for writing to tables, and this owner can only be changed by a workspace admin. 

Here are the steps to change the owner of a Delta Live Tables pipeline:

1. Go to the details page for a pipeline.
2. Click the **Permissions** button in the **Pipeline Details** panel.
3. In the pop-up dialogue box, assign the **Is Owner** permission to the service principal by clicking the drop-down menu beside the service principal's name.
4. Click **Save**.Remember, the service principal must have the necessary permissions for the operations the pipeline is supposed to perform. Please note that once the pipeline owner is set, it cannot be changed for a Unity Catalog (UC)-enabled pipeline. If you need to change the owner for a UC-enabled pipeline, you must create a new pipeline with the desired owner.

Sources:
- [Docs: dlt-acl](https://docs.databricks.com/security/auth-authz/access-control/dlt-acl.html)
- [Docs: unity-catalog](https://docs.databricks.com/delta-live-tables/unity-catalog.html)

knutasm
New Contributor III

Thank you.

However, like you say the owner of the pipeline cannot be changed once set for a UC pipeline. Yet I am unable to choose pipeline owner when creating the pipeline in the first place, which is where I would have to set the service principal. I am also unable to create a pipeline as a service principal.


@knutasm wrote:

Thank you.

However, like you say the owner of the pipeline cannot be changed once set for a UC pipeline. Yet I am unable to choose pipeline owner when creating the pipeline in the first place, which is where I would have to set the service principal. I am also unable to create a pipeline as a service principal.



Hmm.  I wonder if creating the pipeline could it be done with the API?  I've not seen anything about that in my browsing around.  Even if it that's an option, I wouldn't say that's an ideal option.

 

I'm going to assume that DLT with UC is simply not finished baking yet and that this will change sometime...  In the meantime, I might not jump in to DLT with both feet just yet.

Oliver_Angelil
Valued Contributor II

Have the same issue.

serelk
New Contributor III

Same here 

js54123875
New Contributor III

same!

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!