Hi @KevinGagnon, Databricks currently does not have plans to decouple the owner from the "run_as" identity in Delta Live Tables, unlike what can be done with jobs.
The key points are:
- The Delta Live Table pipeline runs using the credentials of the pipeline owner, which means that the owner is also the identity used to run the pipeline.
- There is currently no way to change the "run_as" identity to a service principal, separate from the pipeline owner. This can be an issue when the pipeline should not run as a human user, but rather as a service principal.
- To work around this, the recommended approach is to create a service principal in the Azure Active Directory, add it as a service principal in the Databricks admin settings, and then assign that service principal as the owner of the Delta Live Table pipeline. This allows the pipeline to run using the service principal credentials.
- However, this still means the service principal has full "IS OWNER" permissions on the pipeline, which may provide more access than is required just for running the pipeline.
In summary, Databricks currently does not offer a way to decouple the pipeline owner from the "run_as" identity in Delta Live Tables, unlike the flexibility available for jobs. Users have to work around this by using a dedicated service principal as the pipeline owner. Databricks may consider adding this capability in the future, but there is no indication of any such plans as of now.
