Databricks notebook execution using only one task
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-28-2023 04:33 AM
I am running a databricks notebook. While running, i only see one task on one worker getting started.
My cluster has min 6 workers but seems like they are not getting used.
I am performing a read operation from Cosmos DB.
Can someone please help me here.
Thanks,
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-28-2023 04:43 AM
Hii,
You can adjust graphics settings in your rendering software to a lower quality for smoother playback. And use a higher frame rate for your video, like 60fps, to make transitions smoother.
YourTexasBenefits
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-05-2023 11:05 AM
If your code does not use Spark, it will not use any machines except the driver. If you're using Spark but your source data that you operate on has 1 partition, there will be only 1 task. Hard to say more without knowing what you are doing in more detail
