cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Declative Pipeline: Can pipeline or job be deployed run_as using asset bundle

crami
New Contributor II

Hi, I have very interesting scenario. I am trying to use Declarative pipelines for first time. The platform team has made workspace artefacts as devops based deployment [infra as code], meaning, I cannot create compute. I have to create compute with my login if I want to execute the pipeline.
Now, I am trying to do a workaround. I am actually creating a skeleton pipeline and deploying using asset bundle. I have specified run_as  as a another principal user in pipeline.yml, which do have permissions to create compute. 
After successful deployment, the pipeline still has my name as user. The workspace config mode is 'development' in databricks.yml.
I am looking for a way if I can workaround this issue ? I am trying to avoid requesting platform team to explicitly set run-as on UI.
Thanks you for your kind help in advance.

1 ACCEPTED SOLUTION

Accepted Solutions

donna567taylor
New Contributor

@crami wrote:

Hi, I have very interesting scenario. I am trying to use Declarative pipelines for first time. The platform team has made workspace artefacts as devops based deployment [infra as code], meaning, I cannot create compute. I have to create compute with my login if I want to execute the pipeline.
Now, I am trying to do a workaround. I am actually creating a skeleton pipeline and deploying using asset bundle. I have specified run_as  as a another principal user in pipeline.yml, which do have permissions to create compute. 
After successful deployment, the pipeline still has my name as user. The workspace config mode is 'development' in databricks.yml.
I am looking for a way if I can workaround this issue ? I am trying to avoid requesting platform team to explicitly set run-as on UI.
Thanks you for your kind help in advance.


Hello @crami,
Interesting setup! In development mode, `run_as` in `pipeline.yml` is often ignored—Databricks defaults to the deploying user's identity. To enforce `run_as`, switch workspace config to `production` in `databricks.yml`, which respects the principal override. This avoids needing UI changes and aligns with infra-as-code practices.


Best Regards,
Donna Taylor

View solution in original post

1 REPLY 1

donna567taylor
New Contributor

@crami wrote:

Hi, I have very interesting scenario. I am trying to use Declarative pipelines for first time. The platform team has made workspace artefacts as devops based deployment [infra as code], meaning, I cannot create compute. I have to create compute with my login if I want to execute the pipeline.
Now, I am trying to do a workaround. I am actually creating a skeleton pipeline and deploying using asset bundle. I have specified run_as  as a another principal user in pipeline.yml, which do have permissions to create compute. 
After successful deployment, the pipeline still has my name as user. The workspace config mode is 'development' in databricks.yml.
I am looking for a way if I can workaround this issue ? I am trying to avoid requesting platform team to explicitly set run-as on UI.
Thanks you for your kind help in advance.


Hello @crami,
Interesting setup! In development mode, `run_as` in `pipeline.yml` is often ignored—Databricks defaults to the deploying user's identity. To enforce `run_as`, switch workspace config to `production` in `databricks.yml`, which respects the principal override. This avoids needing UI changes and aligns with infra-as-code practices.


Best Regards,
Donna Taylor

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now