How do I set up a multi-node Intel server cluster for Databricks?

[edisionthomas]

Hello everyone,

I'm looking to set up a multi-node Intel server cluster for Databricks and would appreciate some guidance. Specifically, I want to know the best practices for the initial setup and the necessary configurations to ensure optimal performance.

What are the key considerations for networking, storage, and resource allocation in such a cluster? Additionally, are there any specific Intel server models or configurations that you recommend for this purpose?

Any insights or detailed steps you can share would be incredibly helpful.

[BigRoux]

That is a very open question.  Cluster sizing (classic clusters & warehouses) is very much dependent on the workload (batch processsing, stream processing, machine learning, etc).  Moreover, the amount of data involved plays a very key role (100G, 100T, 1P, etc).  Can you share a bit more about what you are looking to do with this cluster?  Cheers, Louis.