How disable disk cache in SQL Warehouse (Azure Databricks)?

[jlgr]

Hi! I want to disable disk cache for SQL Warehouse in Azure Databricks, but it seems that is not possible. Is it correct?

You can't use this configuration for SQL Warehouse (https://learn.microsoft.com/en-US/azure/databricks/optimizations/disk-cache#--enable-or-disable-the-...), and if you use parameter USED_CACHE_RESULT=false (https://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/parameters/use_cached_result) only is disable Spark cache, not for disk cache.

Could someone confirm it?

Thanks!!

[Kaniz]

Hi @jlgr jlgr​ , To enable and disable the disk cache, run:

spark.conf.set("spark.databricks.io.cache.enabled", "[true | false]")

Disabling the cache does not drop the data already in the local storage. Instead, it prevents queries from adding new data to the store and reading data from the cache.

SOURCE

[jlgr]

Thanks @Kaniz Fatma​! But this options is not available in Databricks SQL (SQL Warehouse). My question is about Databricks SQL.

Is there any options to disable disk cache in Databricks SQL?

[Kaniz · Accepted solution]

Hi @jlgr jlgr​, Databricks SQL supports query caching.

[Anonymous]

Hi @jlgr jlgr​ 

Hope everything is going great.

Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we can help you. 

Cheers!