topic Re: Databricks Asset Bundles - job specific "run_as" user/service_principle in Data Engineering

Databricks Asset Bundles - job specific "run_as" user/service_principle

lilo_z — Tue, 12 Mar 2024 17:12:41 GMT

Was wondering if this was possible, since a use case came up in my team. Would it be possible to use a different service principle for a single job than what is specified for that target environment? For example:

bundle:
  name: hello-bundle

resources:
  jobs:
    hello-job:
      name: hello-job
      tasks:
        - task_key: hello-task
          existing_cluster_id: 1234-567890-abcde123
          notebook_task:
            notebook_path: ./hello.py
    run_as:
      service_principle_name: principle_1

targets:
  dev:
    default: true
    run_as:
       service_principle_name: principle_2

Re: Databricks Asset Bundles - job specific "run_as" user/service_principle

lilo_z — Wed, 13 Mar 2024 22:40:49 GMT

The above configuration isn't working as expected @Retired_mod , is there specific syntax or placement within the yaml file expected within the asset bundle to make the service principle/user explicit? We haven't been able to find documentation that would indicate this to be the case.

Re: Databricks Asset Bundles - job specific "run_as" user/service_principle

lilo_z — Fri, 15 Mar 2024 16:41:23 GMT

Found a working solution, posting it here for anyone else hitting the same issue - trick was to redefine "resources" under the target you want to make an exception for:

bundle: name: hello_bundle include: - resources/*.yml targets: dev: workspace: host: host_name root_path: root_path mode: dev run_as: service_principal_name: DEFAULT_SP resources: jobs: Hello_Job: name: Hello_Job run_as: user_name: SECONDARY_SP/USER