cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Structured Streaming and Workplace Max Jobs

Dekova
New Contributor II

From the documentation: 

  1. A workspace is limited to 1000 concurrent task runs. A 429 Too Many Requests response is returned when you request a run that cannot start immediately.
  2. The number of jobs a workspace can create in an hour is limited to 10000 (includes “runs submit”). This limit also affects jobs created by the REST API and notebook workflows.

If I have a structured streaming pipeline that's triggered every 500ms:

  1. It counts as one task of the 1,000 concurrent task limit?
  2. It can't count as 7,200 jobs created in an hour, how does work with the "jobs created" limit?
0 REPLIES 0

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now