cancel
Showing results for 
Search instead for 
Did you mean: 
Warehousing & Analytics
Engage in discussions on data warehousing, analytics, and BI solutions within the Databricks Community. Share insights, tips, and best practices for leveraging data for informed decision-making.
cancel
Showing results for 
Search instead for 
Did you mean: 

Do SQL Endpoints cache query results?

User16826992666
Valued Contributor
 
1 ACCEPTED SOLUTION

Accepted Solutions

User16826992666
Valued Contributor

SQL Analytics actually uses several layers of caching. Some documentation about the different layers can be found here in the documentation. 

There are two primary layers that users will experience. 

1) The first is that the actual data results of specific queries are stored in memory for subsequent query runs. So if you run the same query twice, it won't have to recompute at all.

2) The second type is delta caching, which is where actual copies of the files that are read from data storage are created on the local SSD's on the endpoint clusters. Then when those files are needed in subsequent queries the cached files on the SSD's are accessed rather than going back and pulling the data from the source again. With this level of caching, you are actually making a copy of the data you are querying on the SSD's of the cluster.

View solution in original post

1 REPLY 1

User16826992666
Valued Contributor

SQL Analytics actually uses several layers of caching. Some documentation about the different layers can be found here in the documentation. 

There are two primary layers that users will experience. 

1) The first is that the actual data results of specific queries are stored in memory for subsequent query runs. So if you run the same query twice, it won't have to recompute at all.

2) The second type is delta caching, which is where actual copies of the files that are read from data storage are created on the local SSD's on the endpoint clusters. Then when those files are needed in subsequent queries the cached files on the SSD's are accessed rather than going back and pulling the data from the source again. With this level of caching, you are actually making a copy of the data you are querying on the SSD's of the cluster.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!