cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

RicksDB
by Contributor II
  • 2069 Views
  • 9 replies
  • 1 kudos

Configure jobs throttling for ephemeral cluster ETLs

Hi,Is it possible to configure job throttling in order to queue jobs across a workspace after a given number of concurrent execution when using the ephemeral cluster pattern? The reason is mainly for cost control. We prefer reducing performance rathe...

  • 2069 Views
  • 9 replies
  • 1 kudos
Latest Reply
RicksDB
Contributor II
  • 1 kudos

Thanks for the help josephk. I will continue to use an interactive cluster for the time being until the release of that new feature. Hopefully, it will allow my use case. Is there visibility on the roadmap for an ETA or more information on it?

  • 1 kudos
8 More Replies
User16776430979
by New Contributor III
  • 1539 Views
  • 1 replies
  • 0 kudos

Repos branch control – how can we configure a job to run a specific branch?

For example, how can we ensure our jobs always run off the main/master branch?

  • 1539 Views
  • 1 replies
  • 0 kudos
Latest Reply
User16781336501
New Contributor III
  • 0 kudos

We recommend having a top level folder to run jobs against. Best practice detailed here: https://docs.databricks.com/repos.html#best-practices-for-integrating-repos-with-cicd-workflows

  • 0 kudos
Labels