cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Attach notebook to cluster via REST API

akihiko
New Contributor III

Is it possible to attach a notebook to cluster and run it via the REST API?

The closest approach I have found is to run a notebook, export the results (HTML!) and import it into the workspace again, but this does not allow us to retain the original execution context. The 1.2 API allows for the creation and manipulation of execution contexts, but I can't figure out how to actual attach a notebook.

My use-case is that we have notebook templates for security incidents that contain some pre-defined cells/queries. We want the system to run these queries before handing the notebook off to an analyst where she can do further analysis.

1 ACCEPTED SOLUTION

Accepted Solutions

Hi @Vivian Wilfred​, I did try that, but that won't allow you to keep the execution context, so it only partially works.

I guess for now I will use this API and export the executed notebook.

View solution in original post

3 REPLIES 3

Ajay-Pandey
Esteemed Contributor III

Hi @Akihiko Nagata​ ,

I think for now we don't have such option available through databricks API.

Vivian_Wilfred
Honored Contributor
Honored Contributor

Hi @Akihiko Nagata​ , have you checked the jobs API? You can run a job on the existing cluster that can use the notebook of concern. I believe this is the only way.

https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsRunsSubmit

Hi @Vivian Wilfred​, I did try that, but that won't allow you to keep the execution context, so it only partially works.

I guess for now I will use this API and export the executed notebook.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!