Hi there,
Can you use a %run or dbutils.notebook.run() in a Delta Live Table (DLT) pipeline?
When I try, I get the following error: "IllegalArgumentException: requirement failed: To enable notebook workflows, please upgrade your Databricks subscription"
Essentially, I would like to have one notebook in the DLT pipeline that calls another notebook. The other "called" notebook creates a Delta Live Table.
For example...
The "caller" notebook, which will be attached to the DLT Pipeline, would look something like this...
dbutils.notebook.run('dlt_test_notebook', 3600)
And the notebook which will be "called" looks something like this...
import dlt
@dlt.table(name = 'test')
def f():
# create dataframe
df = spark.sql(query)
return df
Any help is much appreciated