Databricks Community

knutasm · ‎08-11-2023

How to run a delta live tables pipeline in production? It uses the owner's (creator's) permissions for writing to tables, and I can't change the owner of a UC-enabled pipeline after creation. I don't want regular users to have write access to prod tables, but run as a service principal instead.

I can find no example in the documentation, but it seems like such a glaring shortcoming that I am sure I must have missed something.

knutasm · ‎08-11-2023

Thank you.

However, like you say the owner of the pipeline cannot be changed once set for a UC pipeline. Yet I am unable to choose pipeline owner when creating the pipeline in the first place, which is where I would have to set the service principal. I am also unable to create a pipeline as a service principal.

ilarsen · ‎11-14-2023

@knutasm wrote:
Thank you.
However, like you say the owner of the pipeline cannot be changed once set for a UC pipeline. Yet I am unable to choose pipeline owner when creating the pipeline in the first place, which is where I would have to set the service principal. I am also unable to create a pipeline as a service principal.

Hmm. I wonder if creating the pipeline could it be done with the API? I've not seen anything about that in my browsing around. Even if it that's an option, I wouldn't say that's an ideal option.

I'm going to assume that DLT with UC is simply not finished baking yet and that this will change sometime... In the meantime, I might not jump in to DLT with both feet just yet.