https://community.databricks.com/t5/data-engineering/relation-between-driver-and-executor-size/m-p/11508#M6456 <P>Depends on your use case. The best is to connect Datatog and see driver and workers utilization <A href="https://docs.datadoghq.com/integrations/databricks/?tab=driveronly" target="test_blank">https://docs.datadoghq.com/integrations/databricks/?tab=driveronly</A></P><P></P><P>Just from my experience, Usually, for big datasets, when need autoscale workers between 8 to 32. I set the driver two times bigger</P> Tue, 17 Jan 2023 16:29:21 GMT Hubert-Dudek 2023-01-17T16:29:21Z https://community.databricks.com/t5/data-engineering/relation-between-driver-and-executor-size/m-p/11507#M6455 <P>Hi</P><P></P><P>I would like to ask for recommendations regarding the size of the driver and the amount of executors managed by that driver. I am aware of the best practices regarding executor size/number but I have doubts about the number of executors a single driver can manage. Is it worth investing on a bigger sized driver?</P> Tue, 17 Jan 2023 16:06:05 GMT https://community.databricks.com/t5/data-engineering/relation-between-driver-and-executor-size/m-p/11507#M6455 alvaro_databric 2023-01-17T16:06:05Z https://community.databricks.com/t5/data-engineering/relation-between-driver-and-executor-size/m-p/11508#M6456 <P>Depends on your use case. The best is to connect Datatog and see driver and workers utilization <A href="https://docs.datadoghq.com/integrations/databricks/?tab=driveronly" target="test_blank">https://docs.datadoghq.com/integrations/databricks/?tab=driveronly</A></P><P></P><P>Just from my experience, Usually, for big datasets, when need autoscale workers between 8 to 32. I set the driver two times bigger</P> Tue, 17 Jan 2023 16:29:21 GMT https://community.databricks.com/t5/data-engineering/relation-between-driver-and-executor-size/m-p/11508#M6456 Hubert-Dudek 2023-01-17T16:29:21Z