DLT concurrent pipeline updates.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-01-2023 03:28 AM
Hi!
Regarding this info "An Azure Databricks workspace is limited to 100 concurrent pipeline updates." (Release 2023.16 - Azure Databricks | Microsoft Learn), what is considered as an update? Changes in pipeline logic or each pipeline run?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-04-2023 12:15 AM
First of all, thank you for your reply.
So if I get 99 new files that should trigger DLT pipeline update and at the same time I change something in pipeline logic it means that all these activities contribute to 100 concurrent updates (if I understood correctly).
