DLT Serverless costs
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-30-2024 11:33 PM
I recently started checking out the serverless delta live tables. In my understanding serverless continuous jobs (with autoloader) would only do something when new files arrive. However for serverless 4 pipelines running continuously, I spend in two days almost 2000 euros. Am I doing something wrong? If not, what is the benefit of using serverless continuously against a cheap single node cluster?
- Labels:
-
Workflows
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-31-2024 05:50 PM
Hi @mbaas ,
That does not sound right. So, were you able to compare the jobs, stages and spot the difference. Are there more tasks added, or with compute perspective you do not find a difference at all but only the cost?
Also, it may be required to look at the pipelines closely to understand the reason for increased cost. Is there already an investigation support ticket open for this?
Thanks!
