cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

swzzzsw
by New Contributor III
  • 9380 Views
  • 4 replies
  • 9 kudos

"Run now with different parameters" - different parameters not recognized by jobs involving multiple tasks

I'm running a databricks job involving multiple tasks and would like to run the job with different set of task parameters. I can achieve that by edit each task and and change the parameter values. However, it gets very manual when I have a lot of tas...

  • 9380 Views
  • 4 replies
  • 9 kudos
Latest Reply
VijayNakkonda
New Contributor II
  • 9 kudos

Dear Team, For now, I found a solution. Disconnect the bundle source on Databricks, edit the parameters that you want to run. After execution, redeploy your code again from repository.

  • 9 kudos
3 More Replies
cmilligan
by Contributor II
  • 4313 Views
  • 4 replies
  • 4 kudos

Dropdown for parameters in a job

I want to be able to denote the type of run from a predetermined list of values that a user can choose from when kicking off a run using different parameters. Our team does standardized job runs on a weekly cadence but can have timeframes that change...

  • 4313 Views
  • 4 replies
  • 4 kudos
Latest Reply
Leon_K
New Contributor II
  • 4 kudos

I'm looking to this too. Wonder if there a way to make as a drop down for a job parameter

  • 4 kudos
3 More Replies
Tjadi
by New Contributor III
  • 1859 Views
  • 2 replies
  • 4 kudos

Specifying cluster on running a job

Hi,Let's say that I am starting jobs with different parameters at a certain time each day in the following manner:response = requests.post( "https://%s/api/2.0/jobs/run-now" % (DOMAIN), headers={"Authorization": "Bearer %s" % TOKEN}, json={ ...

  • 1859 Views
  • 2 replies
  • 4 kudos
Latest Reply
karthik_p
Esteemed Contributor
  • 4 kudos

@Tjadi Peeters​ You can select option Autoscaling/Enhanced Scaling in workflows which will scale based on workload

  • 4 kudos
1 More Replies
spott_submittab
by New Contributor II
  • 1136 Views
  • 1 replies
  • 0 kudos

A Job "pool"? (or task pool)

I'm trying to run a single job multiple times with different parameters where the number of concurrent jobs is less than the number of parameters.I have a job (or task...) J that takes parameter set p, I have 100 p values I want to run, however I onl...

  • 1136 Views
  • 1 replies
  • 0 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 0 kudos

this is something new,interesting question, try to reach out databricks support team, maybe they have some good idea here

  • 0 kudos
Labels