Hello,I would like to use job parameters in spark_python_task (not notebook_task), does anyone know how to retrieve these parameters inside pure Python ?I tried:1/ dbutils.widgets.get("debug"), got error: com.databricks.dbutils_v1.InputWidgetNotD...
Hello,I've read the posts:Jobs - Delta Live tables difference (databricks.com)andDifference between Delta Live Tables and Multitask Jobs (databricks.com)My understanding is that delta live tables are more like a DSL that simplfies the workflow defini...
there're 2 solutions:official one: Re: Retrieve job-level parameters in Python - Databricks Community - 44720another one is to use the jobs rest api.The official one is only available once you enter the argparse part (for some use cases, it might be ...
A workaround that I found is to use the databricks jobs api to get the job run info. There are job parameters inside, but you need to prepare crendetials in advance.
