- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-06-2022 11:37 AM
Hi there,
Can you use a %run or dbutils.notebook.run() in a Delta Live Table (DLT) pipeline?
When I try, I get the following error: "IllegalArgumentException: requirement failed: To enable notebook workflows, please upgrade your Databricks subscription"
Essentially, I would like to have one notebook in the DLT pipeline that calls another notebook. The other "called" notebook creates a Delta Live Table.
For example...
The "caller" notebook, which will be attached to the DLT Pipeline, would look something like this...
dbutils.notebook.run('dlt_test_notebook', 3600)
And the notebook which will be "called" looks something like this...
import dlt
@dlt.table(name = 'test')
def f():
# create dataframe
df = spark.sql(query)
return df
Any help is much appreciated
- Labels:
-
Dbutils
-
DLT
-
Run
-
Table Pipeline
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-10-2022 01:31 AM
Hi all.
@Kaniz Fatma thanks for your answer. I am on the premium pricing tier in Azure.
After digging around the logs it would seem that you cannot run magic commands in a Delta Live Table pipeline. Therefore, you cannot use %run in a DLT pipeline - which is a shame 😞
Hope this helps someone else who is struggling with this!
Thanks, JMW
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-10-2022 01:31 AM
Hi all.
@Kaniz Fatma thanks for your answer. I am on the premium pricing tier in Azure.
After digging around the logs it would seem that you cannot run magic commands in a Delta Live Table pipeline. Therefore, you cannot use %run in a DLT pipeline - which is a shame 😞
Hope this helps someone else who is struggling with this!
Thanks, JMW
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-02-2023 07:06 AM
Does that mean you can run dbutils command?

