If data from a Delta table is cached in Databricks SQL and the table is altered in the backend, does it invalidate the cache?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-15-2021 02:17 PM
Basically I'm worried about the scenario where data that gets cached on Databricks SQL endpoints becomes out of sync with the source Delta table. If that were to happen and data was read from the cache it would be out of date/incorrect.
Is this a concern? Is there a way that the Databricks SQL endpoint will know the cache needs to be refreshed?
- Labels:
-
Delta
-
Delta Cache
-
SQL
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-20-2021 02:31 PM
There are 3 types of caching.
1-Databricks SQL UI caching,
2-Query results caching ,
3-Delta caching .
(1) does not get invalidated. It's like your BI dashboard. BI dashboard needs to be manually refreshed.
(2) and (3) gets auto invalidation.
pls check for more info

