topic Re: Databricks notebook execution using only one task in Get Started Discussions

Databricks notebook execution using only one task

ankit_batra1 — Mon, 28 Aug 2023 11:33:22 GMT

I am running a databricks notebook. While running, i only see one task on one worker getting started.

My cluster has min 6 workers but seems like they are not getting used.

I am performing a read operation from Cosmos DB.

Can someone please help me here.

Thanks,

Re: Databricks notebook execution using only one task

Jaytalbot — Mon, 28 Aug 2023 11:43:06 GMT

Hii,

You can adjust graphics settings in your rendering software to a lower quality for smoother playback. And use a higher frame rate for your video, like 60fps, to make transitions smoother.

Re: Databricks notebook execution using only one task

sean_owen — Tue, 05 Sep 2023 18:05:38 GMT

If your code does not use Spark, it will not use any machines except the driver. If you're using Spark but your source data that you operate on has 1 partition, there will be only 1 task. Hard to say more without knowing what you are doing in more detail