Delay in retrieving task variables from a notebook utilizing %run
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-09-2023 07:32 AM
I have a databricks workflow where the first task is to set up task parameters for other notebooks to use later in the process. Since these are variables that are used in all of my notebooks, I have opted to assign them in a shared notebook and call that notebook using %run.
In the tasks immediately following the the notebook where I am setting the variables and assigning them I'm getting failures in the %run command due to the variables not being set. I've tried using time.sleep() in the notebook that I'm setting the variables but that hasn't worked. I never had this issue when calling them directly in the notebook, just when calling them in a separate notebook using %run.
Is there a known issue where there is a delay between when a variable is set and how soon after it can be called.
- Labels:
-
Run
-
Task Parameters
-
Task Variables
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-10-2023 11:29 AM
Hi @Coleman Milligan , How are you setting the variables in the shared notebook? Could you please share some code to understand better?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-15-2023 09:40 AM
Hi, Please refer: https://docs.databricks.com/notebooks/notebook-workflows.html
Could you please provide the error received?