cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Multiple schedules in workflow with different parameters

Alby091
New Contributor

I have a notebook that takes a file from the landing, processes it and saves a delta table.
This notebook contains a parameter (time_prm) that allows you to do this option for the different versions of files that arrive every day.

Specifically, for each file, I will have 3 daily versions and the nomenclature is the following file_202401070900, file_202401071200, file_202401071500.

I have a workflow that launches this notebook but I need to schedule it 3 times a day and for each build of the job, I will need to take the relative file.

For example, in the 0900 schedule I will need to take the file version file_202401070900, so I will insert 0900 as the time_prm parameter. For the file file_202401071200 I will insert 1200 etc.

Can I create multiple schedules of the same workflow and set a different time parameter for each schedule?

Do you have any other ideas? It seems strange to me that I have to create 3 different workflows.

Thanks for your help!

1 REPLY 1

Walter_C
Databricks Employee
Databricks Employee

Right now jobs support only 1 Schedule process per job, as you have mentioned you will need to create a different job for each schedule you require, you can use the clone capability to facilitate the process.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group