How do I know which SQL Endpoint size to choose?

User16826992666 · ‎06-15-2021

Digan_Parikh · ‎06-22-2021

In general, we can say that bigger clusters will process big queries faster and improve throughput. You should keep in mind that bigger clusters will not help in serving more concurrent queries. Endpoints will auto-scale (add a new cluster) when processing approximately 10 parallel requests.

Digan_Parikh · ‎06-22-2021

When sizing, this is the recommendation.

Data set Cluster Size

ITB / rows X-Large+

500GB / 1B rows X-Large

SOGB / IOOM+ rows Large

IOOGB / rows Medium

IOGB / -M rows Small

This table maps SQL endpoint cluster sizes to Databricks cluster driver sizes and worker counts. All workers are i3.2xlarge.

View solution in original post