cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Declative Pipeline: Can pipeline or job be deployed run_as using asset bundle

crami
New Contributor

Hi, I have very interesting scenario. I am trying to use Declarative pipelines for first time. The platform team has made workspace artefacts as devops based deployment [infra as code], meaning, I cannot create compute. I have to create compute with my login if I want to execute the pipeline.
Now, I am trying to do a workaround. I am actually creating a skeleton pipeline and deploying using asset bundle. I have specified run_as  as a another principal user in pipeline.yml, which do have permissions to create compute. 
After successful deployment, the pipeline still has my name as user. The workspace config mode is 'development' in databricks.yml.
I am looking for a way if I can workaround this issue ? I am trying to avoid requesting platform team to explicitly set run-as on UI.
Thanks you for your kind help in advance.

1 REPLY 1

donna567taylor
New Contributor

@crami wrote:

Hi, I have very interesting scenario. I am trying to use Declarative pipelines for first time. The platform team has made workspace artefacts as devops based deployment [infra as code], meaning, I cannot create compute. I have to create compute with my login if I want to execute the pipeline.
Now, I am trying to do a workaround. I am actually creating a skeleton pipeline and deploying using asset bundle. I have specified run_as  as a another principal user in pipeline.yml, which do have permissions to create compute. 
After successful deployment, the pipeline still has my name as user. The workspace config mode is 'development' in databricks.yml.
I am looking for a way if I can workaround this issue ? I am trying to avoid requesting platform team to explicitly set run-as on UI.
Thanks you for your kind help in advance.


Hello @crami,
Interesting setup! In development mode, `run_as` in `pipeline.yml` is often ignoredโ€”Databricks defaults to the deploying user's identity. To enforce `run_as`, switch workspace config to `production` in `databricks.yml`, which respects the principal override. This avoids needing UI changes and aligns with infra-as-code practices.


Best Regards,
Donna Taylor