Run an Azure Databricks notebook from another notebook with ipywidget

Question

I am trying to run a notebook from another notebook using the dbutils.notebook.run as follows:

import ipywidgets as widgets
from ipywidgets import interact
from ipywidgets import Box

button = widgets.Button(description='Run model')
out = widgets.Output()
def on_button_clicked(b):
    button.description = 'Run model'
    with out:
        dbutils.notebook.run("/mynotebookpath",60)

button.on_click(on_button_clicked)
widgets.VBox([button, out])

However, I am getting the following error:

IllegalArgumentException: Context not valid. If you are calling this outside the main thread, you must set the Notebook context via dbutils.notebook.setContext(ctx), where ctx is a value retrieved from the main thread (and the same cell)

I can run the notebook just fine when I do %run on a single cell and even dbutils.notebook.run("/mynotebook", 60) on a single cell. However I cannot get it to run within the ipywidget context

Answer 1

When I executed the given code, I have got the same error. However, there is no way to follow the error message to use dbutils.notebook.setContext as there is no setContext attribute.

• Using dbutils.notebook.run also creates an untitled job run. So, as an alternative you can use Jobs 2.0 to run the notebook.
• The following is the code that you can use:

import ipywidgets as widgets
import requests

response = None

def my_function(param):
    print(f"This is {param}")
    global response

    my_json = {"notebook_task": {"notebook_path": "/nb2"},"run_name": "execute notebook","timeout_seconds": 3600,"existing_cluster_id": "<cluster_id>"}   
    auth = {"Authorization": "Bearer <access_token>"}
    response = requests.post('https://<workspace_instance>/api/2.0/jobs/runs/submit', json = my_json, headers=auth).json()
    print(response)

button = widgets.Button(description="Run Model")
output = widgets.Output()

display(button, output)

def on_button_click(b):
    with output:
        print("button is clicked")
        my_function('in fuction')

button.on_click(on_button_click)

• You can monitor this run and check for its execution. The following is the sample run of my notebook.