topic Re: Running notebook from another notebook does not work when running notebook from github actions in Data Engineering

Running notebook from another notebook does not work when running notebook from github actions

Ullsokk — Thu, 26 Jan 2023 14:36:58 GMT

I have a setup-notebook that users %run to run a series of notebooks. The notebook is in the root folder of my repo. In a subfolder I have several notebooks I want to run. If I run the notebook in databricks, the relative paths work (%run "./subfolder/some notebook"). But when I try to run this setup-notebook from github actions, I get the following error: Notebook not found: tmp/databricks-github-actions/24245-645-tr-rtgtr-3443/subfolder/some notebook.

Re: Running notebook from another notebook does not work when running notebook from github actions

Ajay-Pandey — Thu, 26 Jan 2023 16:41:55 GMT

Hi @Stian Arntsen I think for now databricks does not support notebook run from github action.

Re: Running notebook from another notebook does not work when running notebook from github actions

Hubert-Dudek — Thu, 26 Jan 2023 16:44:35 GMT

It seems that the directory is different under GitHub actions. Not sure what exactly mentioned GitHub actions do and which one it is. Maybe you can share GitHub action used.

Re: Running notebook from another notebook does not work when running notebook from github actions

Ullsokk — Thu, 26 Jan 2023 20:15:05 GMT

I can run other notebooks just fine from github actions. But running a notebook that runs another notebook doesn't seem to work for me.

Re: Running notebook from another notebook does not work when running notebook from github actions

Ullsokk — Thu, 26 Jan 2023 20:17:30 GMT

name: Setup
on: 
  workflow_dispatch:
jobs:
  run-notebooks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Run a databricks notebook
        uses: databricks/run-notebook@v0
        with:
         local-notebook-path: Setup.sql
         databricks-host: https://myhost.cloud.databricks.com/
         databricks-token: ${{ secrets.DATABRICKS_TOKEN }}
         git-commit: ${{ github.event.pull_request.head.sha }}
         existing-cluster-id : myclusterid

This is the yml file running from github actions.