Hi @jlgr jlgr , To enable and disable the disk cache, run:
spark.conf.set("spark.databricks.io.cache.enabled", "[true | false]")
Disabling the cache does not drop the data already in the local storage. Instead, it prevents queries from adding new data to the store and reading data from the cache.
SOURCE