https://community.databricks.com/t5/data-engineering/what-is-the-local-ssd-used-for-in-databricks/m-p/53309#M29779 <P>What is the use-case for local-ssd's in databricks clusters? I noticed some clusters have many Tb's worth and some have no local ssd's.</P><P>What are the pro's and con's of changing the disk size bigger and smaller?</P><P>&nbsp;</P><P>According to the docs:</P><P>&gt; The disk cache is configured to use at most half of the space available on the local SSDs provided with the worker nodes. For configuration options, see <A href="https://docs.gcp.databricks.com/optimizations/disk-cache.html#configure-cache" target="_blank" rel="noopener"><SPAN class="">Configure the disk cache</SPAN></A>.</P><P>But what is the other half used for? Is it swap? Sorting tempfiles written during execution.</P> Tue, 21 Nov 2023 15:55:59 GMT 646901 2023-11-21T15:55:59Z https://community.databricks.com/t5/data-engineering/what-is-the-local-ssd-used-for-in-databricks/m-p/53309#M29779 <P>What is the use-case for local-ssd's in databricks clusters? I noticed some clusters have many Tb's worth and some have no local ssd's.</P><P>What are the pro's and con's of changing the disk size bigger and smaller?</P><P>&nbsp;</P><P>According to the docs:</P><P>&gt; The disk cache is configured to use at most half of the space available on the local SSDs provided with the worker nodes. For configuration options, see <A href="https://docs.gcp.databricks.com/optimizations/disk-cache.html#configure-cache" target="_blank" rel="noopener"><SPAN class="">Configure the disk cache</SPAN></A>.</P><P>But what is the other half used for? Is it swap? Sorting tempfiles written during execution.</P> Tue, 21 Nov 2023 15:55:59 GMT https://community.databricks.com/t5/data-engineering/what-is-the-local-ssd-used-for-in-databricks/m-p/53309#M29779 646901 2023-11-21T15:55:59Z