Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
Since Workflows (Multi-Task jobs) is now GA, one way to work around the 1000 concurrent jobs limit is to use tasks within a job. Each job can have 100 tasks, and these tasks do not count toward the concurrent job limit.
Since Workflows (Multi-Task jobs) is now GA, one way to work around the 1000 concurrent jobs limit is to use tasks within a job. Each job can have 100 tasks, and these tasks do not count toward the concurrent job limit.
Join Us as a Local Community Builder!
Passionate about hosting events and connecting people? Help us grow a vibrant local communityโsign up today to get started!
