cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks Asset bundle

Aria
New Contributor III

Hi,

I am new to databricks, We are trying to use Databricks asset bundles for code deployment .I have spect a lot of time but still so many things are not clear to me.

Can we change the target path of the notebooks deployed from /shared/.bundle/* to something like a shared folder in workspace or a repos in higher environment?

All the examples are to deploy jobs, delta live tables etc. How can I deploy a simple notebook without any job.

 

#databricksassetbundle #asset #deployment #bundle #CICD

 

3 REPLIES 3

Kaniz
Community Manager
Community Manager

Hi @Aria, Let’s break down your questions about Databricks Asset Bundles:

 

Changing the Target Path:

  • By default, Databricks Asset Bundles deploy to the /shared/.bundle/* path. However, you can customize the target path to a shared folder in the workspace or a repository in a higher environment.
  • To achieve this, you need to define a new target in your bundle configuration file (usually in YAML format). For example:targets:  my_custom_target:    mode: development  # Or 'production' based on your use case    # Add other relevant configurations here
  • Replace my_custom_target with your desired target name and adjust the mode (development or production) accordingly.
  • You can then deploy your bundle to this custom target using the databricks bundle deploy -t my_custom_target command.

Deploying a Simple Notebook:

  • To deploy a notebook without any associated job, follow these steps:
    • Create a Databricks Asset Bundle configuration file (e.g., my_bundle.yaml) with the necessary settings.
    • Define the notebook(s) you want to include in the bundle:notebooks:  - path: /Workspace/MyNotebook    # Add other relevant notebook properties here
    • Run the deployment command:databricks bundle deploy -t my_custom_target -c my_bundle.yaml
    • Replace /Workspace/MyNotebook with the actual path to your notebook.
    • The notebook will be deployed according to the specified target and mode.

Remember that Databricks Asset Bundles provide flexibility for managing your workflows, and you can tailor them to your specific needs. If you encounter any issues or need further clarification, feel free to ask! 🚀

Aria
New Contributor III

Thank you @Kaniz for responding. 

I have tried to use a custom target also.I am using Databricks asset bundle from my local machine and have not configured it in azure devops.

Lets say I use the below custom path in target.

 root_path: /Shared/Admin
I get below folder created under /Shared/Admin after deployment and my notebook is placed in resources folder in files as in my visual studio code its under resources folder. I provided a nootbook path but that did not help.Can I just deploy my notebook inside Admin folder?
 
Aria_0-1702047979208.png

 

 

 

 

 

Еmil
New Contributor III

Hi @Kaniz,

Thank you for your post, I thought it will solve my issues too, however after reading your suggestion it was nothing new for me, because l already have done exactly that.. Here is what I have dome so you or anyone can replicate it:

1. I used databricks bundle init to create a bundle with default python template 
2. I modified the generated databicks.yaml to have my custom target names a per your post.

When I run databricks bundle deploy -t my_env I have set the Repo main branch for both production an development mode environments. I expect when deploying the code from the main branch repo to be used however I can not get that working...

Here is the error I get:

databricks bundle deploy -t dev_001 --var="shared_autoscaling_cluster_id=****"
Starting upload of bundle files
Error: /Repos/Actuarial/CLUKDataPlatform.Bulkmarker.Notebooks/files does not exist; please create it first

Here is my bundle yaml for the custom target:
targets:
    mode: production
    git:
      branch: main
    default: true
    workspace:
      root_path: /Repos/MyProject/AzDO.Repo.Notebooks

Here is my resource basic notebook yaml:
resources:
  jobs:
    A_basic_inputs:
      name: A_basic_inputs
      deployment_config:
        no_package: true
      git_source:
        git_provider: azureDevOpsServices
        git_branch: main
      tasks:
        - task_key: A_basic_inputs
          notebook_task:
            notebook_path: subfolder/A_basic_inputs.py
            source: WORKSPACE
          existing_cluster_id: ${var.shared_autoscaling_cluster_id}

Please, please can someone tell me what is wrong and how do I configure the yaml to point to the Repo?
Any pointers to working code example or documentation/schema yaml reference is much appreciated.

Thanks!

Emil

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!