Schedule job runs with different parameters?

[129876]

Is it possible to schedule different runs for job with parameters? I have a notebook that generates data based on the supplied parameter but would like to schedule runs instead of manually starting them. I assume that this would be possible using the REST API but would like to avoid having my token in plaintext. Would git integration have something for this?

[Debayan]

You can pass parameters for your task. Each task type has different requirements for formatting and passing the parameters.

https://docs.databricks.com/workflows/jobs/jobs.html#create-a-job

REST API can also pass parameters fro jobs. Tokens replace passwords in an authentication flow and should be protected like passwords. To protect tokens, Databricks recommends that you store tokens in:

• Secret management and retrieve tokens in notebooks using the Secrets utility (dbutils.secrets).
• A local key store and use the Python keyring package to retrieve tokens at runtime.

Please refer: https://docs.databricks.com/dev-tools/api/latest/authentication.html

Also, you can store the tokens in .netrc file and use them in curl : https://docs.databricks.com/dev-tools/api/latest/authentication.html#store-tokens-in-a-netrc-file-an...

For GIT integration please refer: https://docs.databricks.com/repos/index.html.

Please let us know if this helps.

[129876]

Yes I am aware of these. I wanted to know if I can schedule tasks with different parameters, is there any functionality in Databricks that would allow this?

[129876]

I want to schedule this process daily. To run automatically.

[129876]

Not really... I have different values for a single parameter within a notebook. Can I schedule under a single job, runs for each different value?