- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-19-2023 08:56 AM
Assuming by workflow, you mean jobs then yes for parameters using the jobs api
https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/2.0/jobs
Scheduling is a different matter. This may be possible through the spark_submit_params that the API exposes but not something i have tried. I ended up creating a scheduling engine outside of Databricks which is called by Data Factory which works out what to execute at a given invocation time
The parameter values to be provided to the Jobs API are passed from the scheduling engine into Databricks by Data Factory which will then override the parameter values stored at the Job level.
I am also assuming the underlying code is generic and will run based on the parameter values provided...