https://community.databricks.com/t5/data-engineering/how-to-update-num-workers-dynamically-in-a-job-cluster/m-p/81206#M36251 <P>Thank you! Will give it a try.</P> Tue, 30 Jul 2024 16:59:09 GMT NickLee 2024-07-30T16:59:09Z https://community.databricks.com/t5/data-engineering/how-to-update-num-workers-dynamically-in-a-job-cluster/m-p/80754#M36149 <P>I am setting up a workflows with the UI. In the first task, a dynamic value for the next task's num_workers is calculated based on actual data size. In the subsequent task, I'd like to use this calculated num_workers to update the job cluster's default num_workers. Is this possible? Thanks.</P><P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="NickLee_0-1722018584496.png" style="width: 400px;"><img src="https://community.databricks.com/t5/image/serverpage/image-id/9928i240A5546D230A97B/image-size/medium/is-moderation-mode/true?v=v2&amp;px=400" role="button" title="NickLee_0-1722018584496.png" alt="NickLee_0-1722018584496.png" /></span></P><P>&nbsp;</P> Fri, 26 Jul 2024 18:32:51 GMT https://community.databricks.com/t5/data-engineering/how-to-update-num-workers-dynamically-in-a-job-cluster/m-p/80754#M36149 NickLee 2024-07-26T18:32:51Z https://community.databricks.com/t5/data-engineering/how-to-update-num-workers-dynamically-in-a-job-cluster/m-p/81002#M36193 <P>wonder if anyone has similar experience? thanks</P> Mon, 29 Jul 2024 17:27:16 GMT https://community.databricks.com/t5/data-engineering/how-to-update-num-workers-dynamically-in-a-job-cluster/m-p/81002#M36193 NickLee 2024-07-29T17:27:16Z https://community.databricks.com/t5/data-engineering/how-to-update-num-workers-dynamically-in-a-job-cluster/m-p/81206#M36251 <P>Thank you! Will give it a try.</P> Tue, 30 Jul 2024 16:59:09 GMT https://community.databricks.com/t5/data-engineering/how-to-update-num-workers-dynamically-in-a-job-cluster/m-p/81206#M36251 NickLee 2024-07-30T16:59:09Z