cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks asset bundles deployment to development

naga_databricks
Contributor

Hi All,

I am using Databricks Asset Bundles to deploy my code on github to databricks workspace. I have written out the Github Action as provided on databricks documentation.

I have setup the personal access token for the service principal I want to use to deploy with a Github Action.

I have can_user & can_manage permisions on this service principal.

I generated a personal access token for a service principal following the steps from https://docs.gcp.databricks.com/dev-tools/service-principals.html

Following which I have setup the Github Action following https://docs.gcp.databricks.com/dev-tools/bundles/ci-cd.html

I have the SP_TOKEN setup on the github secrets for the respective environments.

 

When I deploy from my local using my user personal access token, I can see the development mode deployment working very well.

However, on using my user personal token or the personal token of service principal from Github Actions, I get following error:

Error: default auth: cannot configure default credentials. Config: host=

 

Following is the Github Actions:

 

# This workflow validates, deploys, and runs the specified bundle
# within a pre-production target named "qal".
name: "DAB - Non Production deployment"

# Ensure that only a single job or workflow using the same concurrency group
# runs at a time.
concurrency: "1"

## Trigger this workflow whenever a pull request is opened against the repo's
## main branch or an existing pull request's head branch is updated.
on:
  workflow_dispatch:
    inputs:
      Environment:
        type: choice
        description: 'Choose the environment:'
        required: true
        options:
          - dev
          - qal


jobs:
  deploy:
    name: "Deploy bundle"
    runs-on: ubuntu-latest

    steps:
      # Check out this repo, so that this workflow can access it.
      - uses: actions/checkout@v3

      # Download the Databricks CLI.
      # See https://github.com/databricks/setup-cli
      - uses: databricks/setup-cli@main


      # Validate the bundle to the selected target as defined
      # in the bundle's settings file.
      - run: databricks bundle validate
        working-directory: .
        env:
          DATABRICKS_TOKEN: ${{ secrets.SP_TOKEN }}
          DATABRICKS_BUNDLE_ENV: ${{github.event.inputs.Environment}}


      # Deploy the bundle to the selected target as defined
      # in the bundle's settings file.
      - run: databricks bundle deploy
        working-directory: .
        env:
          DATABRICKS_TOKEN: ${{ secrets.SP_TOKEN }}
          DATABRICKS_BUNDLE_ENV: ${{github.event.inputs.Environment}}

 

Can someone who have setup the Github Actions with DAB help me out

 

1 ACCEPTED SOLUTION

Accepted Solutions

naga_databricks
Contributor

Finally, i was able to identify the missing piece. This was setting up the environment identifier for the runner. 

    name: "Deploy bundle"
    runs-on: ubuntu-latest
    environment: ${{github.event.inputs.Environment}}

With this, the action was able to get the secrets from the github environment. 

The host property for DATABRICKS_HOST is provided via the bundles configurations, it doesnt need to be provided again on the Github action.

View solution in original post

1 REPLY 1

naga_databricks
Contributor

Finally, i was able to identify the missing piece. This was setting up the environment identifier for the runner. 

    name: "Deploy bundle"
    runs-on: ubuntu-latest
    environment: ${{github.event.inputs.Environment}}

With this, the action was able to get the secrets from the github environment. 

The host property for DATABRICKS_HOST is provided via the bundles configurations, it doesnt need to be provided again on the Github action.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group