Run a Databricks notebook from another notebook with ipywidget

[mbejarano89]

0

I am trying to run a notebook from another notebook using the dbutils.notebook.run as follows:

import ipywidgets as widgets

from ipywidgets import interact

from ipywidgets import Box

button = widgets.Button(description='Run model')

out = widgets.Output()

def on_button_clicked(b):

button.description = 'Run model'

with out:

dbutils.notebook.run("/mynotebookpath",60)

button.on_click(on_button_clicked)

widgets.VBox([button, out])

However, I am getting the following error:

IllegalArgumentException: Context not valid. If you are calling this outside the main thread, you must set the Notebook context via dbutils.notebook.setContext(ctx), where ctx is a value retrieved from the main thread (and the same cell)

I can run the notebook just fine when I do 

%run

 on a single cell and even 

dbutils.notebook.run("/mynotebook", 60)

 on a single cell. However I cannot get it to run within the ipywidget context

[Anonymous]

@Marcela Bejarano​ :

The error you are seeing is because the dbutils module requires a valid notebook context to run the notebook. When running code from an ipywidget context, the main thread and the widget thread are different. To solve this, you can try setting the notebook context before calling dbutils.notebook.run()

import ipywidgets as widgets
from ipywidgets import interact, Box
from pyspark.sql import SparkSession
 
button = widgets.Button(description='Run model')
out = widgets.Output()
 
def on_button_clicked(b):
    button.description = 'Run model'
    with out:
        # Get the current notebook context
        current_notebook = get_ipython().get_parent()
        # Set the notebook context for dbutils
        dbutils.notebook.setContext(current_notebook)
        # Run the notebook
        dbutils.notebook.run("/mynotebookpath",60)
 
button.on_click(on_button_clicked)
widgets.VBox([button, out])

This should set the notebook context correctly and allow you to run the notebook from the ipywidget context.

[mbejarano89]

Hello,

Thank you for this. I tried running it but now I am getting the error:

AttributeError: 'NotebookHandler' object has no attribute 'setContext'

[TheDarkKnight]

Hey !

Have you found a solution for that isssue?
I am facing the same problem...
"AttributeError: 'NotebookHandler' object has no attribute 'setContext'"

Thanks in advance!

[Anonymous]

Hi @Marcela Bejarano​ 

I'm sorry you could not find a solution to your problem in the answers provided.

Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.

I suggest providing more information about your problem, such as specific error messages, error logs or details about the steps you have taken. This can help our community members better understand the issue and provide more targeted solutions.

Alternatively, you can consider contacting the support team for your product or service. They may be able to provide additional assistance or escalate the issue to the appropriate section for further investigation.

Thank you for your patience and understanding, and please let us know if there is anything else we can do to assist you.

[Sreekanth_N]

I am also getting the same error

[Sreekanth_N]

As I could see the pyspark stream is not supporting this setContext, ideally it should have alternative approach. please suggest what is approach where pyspark stream is internally calling to another notebook parallel