- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-25-2025 11:44 PM
I am wondering if it is using Remote result cache, in that case the config should work.
There are 4 types of cache mentioned here https://docs.databricks.com/en/sql/user/queries/query-caching.html#types-of-query-caches-in-databric...
Local cache: once the cluster is stopped or restarted, the cache is cleaned and all query results are removed.
Databricks SQL UI cache: the cache is invalidated once the underlying tables have been updated.
You can delete query results by re-running the query that you no longer want to be stored. Once re-run, the old query results are removed from cache.
Disk cache: stores data on disk, allowing for accelerated data reads.
disk cache automatically detects changes to the underlying data files. When it detects changes, the cache is invalidated. The disk cache shares the same lifecycle characteristics as the local result cache. This means that when the cluster is stopped or restarted, the cache is cleaned and needs to be repopulated.
Even when I stop my serverless warehouse cluster and start it again it uses cache.
So, that only leaves us to Remote result cache : persisting them as workspace system data. This cache is not invalidated by the stopping or restarting of a SQL warehouse.
SET use_cached_result = false;