cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Run a Databricks notebook from another notebook with ipywidget

mbejarano89
New Contributor III

0

I am trying to run a notebook from another notebook using the dbutils.notebook.run as follows:

import ipywidgets as widgets

from ipywidgets import interact

from ipywidgets import Box

button = widgets.Button(description='Run model')

out = widgets.Output()

def on_button_clicked(b):

button.description = 'Run model'

with out:

dbutils.notebook.run("/mynotebookpath",60)

button.on_click(on_button_clicked)

widgets.VBox([button, out])

However, I am getting the following error:

IllegalArgumentException: Context not valid. If you are calling this outside the main thread, you must set the Notebook context via dbutils.notebook.setContext(ctx), where ctx is a value retrieved from the main thread (and the same cell)

I can run the notebook just fine when I do 

%run

 on a single cell and even 

dbutils.notebook.run("/mynotebook", 60)

 on a single cell. However I cannot get it to run within the ipywidget context

6 REPLIES 6

Anonymous
Not applicable

@Marcela Bejarano​ :

The error you are seeing is because the dbutils module requires a valid notebook context to run the notebook. When running code from an ipywidget context, the main thread and the widget thread are different. To solve this, you can try setting the notebook context before calling dbutils.notebook.run()

import ipywidgets as widgets
from ipywidgets import interact, Box
from pyspark.sql import SparkSession
 
button = widgets.Button(description='Run model')
out = widgets.Output()
 
def on_button_clicked(b):
    button.description = 'Run model'
    with out:
        # Get the current notebook context
        current_notebook = get_ipython().get_parent()
        # Set the notebook context for dbutils
        dbutils.notebook.setContext(current_notebook)
        # Run the notebook
        dbutils.notebook.run("/mynotebookpath",60)
 
button.on_click(on_button_clicked)
widgets.VBox([button, out])

This should set the notebook context correctly and allow you to run the notebook from the ipywidget context.

Hello,

Thank you for this. I tried running it but now I am getting the error:

AttributeError: 'NotebookHandler' object has no attribute 'setContext'

Hey !

Have you found a solution for that isssue?
I am facing the same problem...
"AttributeError: 'NotebookHandler' object has no attribute 'setContext'"

Thanks in advance!

Anonymous
Not applicable

Hi @Marcela Bejarano​ 

I'm sorry you could not find a solution to your problem in the answers provided.

Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.

I suggest providing more information about your problem, such as specific error messages, error logs or details about the steps you have taken. This can help our community members better understand the issue and provide more targeted solutions.

Alternatively, you can consider contacting the support team for your product or service. They may be able to provide additional assistance or escalate the issue to the appropriate section for further investigation.

Thank you for your patience and understanding, and please let us know if there is anything else we can do to assist you.

Sreekanth_N
New Contributor II

I am also getting the same error

Sreekanth_N
New Contributor II

As I could see the pyspark stream is not supporting this setContext, ideally it should have alternative approach. please suggest what is approach where pyspark stream is internally calling to another notebook parallel

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!