cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Warehousing & Analytics
Engage in discussions on data warehousing, analytics, and BI solutions within the Databricks Community. Share insights, tips, and best practices for leveraging data for informed decision-making.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Do SQL Endpoints cache query results?

User16826992666
Valued Contributor
 
1 ACCEPTED SOLUTION

Accepted Solutions

User16826992666
Valued Contributor

SQL Analytics actually uses several layers of caching. Some documentation about the different layers can be found here in the documentation. 

There are two primary layers that users will experience. 

1) The first is that the actual data results of specific queries are stored in memory for subsequent query runs. So if you run the same query twice, it won't have to recompute at all.

2) The second type is delta caching, which is where actual copies of the files that are read from data storage are created on the local SSD's on the endpoint clusters. Then when those files are needed in subsequent queries the cached files on the SSD's are accessed rather than going back and pulling the data from the source again. With this level of caching, you are actually making a copy of the data you are querying on the SSD's of the cluster.

View solution in original post

1 REPLY 1

User16826992666
Valued Contributor

SQL Analytics actually uses several layers of caching. Some documentation about the different layers can be found here in the documentation. 

There are two primary layers that users will experience. 

1) The first is that the actual data results of specific queries are stored in memory for subsequent query runs. So if you run the same query twice, it won't have to recompute at all.

2) The second type is delta caching, which is where actual copies of the files that are read from data storage are created on the local SSD's on the endpoint clusters. Then when those files are needed in subsequent queries the cached files on the SSD's are accessed rather than going back and pulling the data from the source again. With this level of caching, you are actually making a copy of the data you are querying on the SSD's of the cluster.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group