cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

use_cached_result

angel_ba
New Contributor II

i am trying to execute same query on 3 different platforms - dbeaver, python notebook and sql workflow.I was expecting after first execution of the query irrespective of the platform, subsequent execution of same query should NOT re-compute. However each platform behaves independently. I assume use_cached_result = true by default. How can I use the results of the query executed once across all the platforms

1 ACCEPTED SOLUTION

Accepted Solutions

Kaniz_Fatma
Community Manager
Community Manager

Hi @bhawana, One possible solution is to use a caching layer in front of your database, such as Memcached or SafePeak. These tools can store the query results in a distributed memory cache and allow you to access them from multiple platforms. This way, you can avoid hitting the database directly for each query and reduce the load on your database server.

 

Another possible solution is to use a temporary table or a view in your database to store the query results. You can then join this table or view it with other tables or views in your queries and avoid repeating the same query. This way, you can leverage the indexes and statistics of your database to optimize your queries.

 

You can find more information about these solutions :

I hope this helps you with your problem. Please let me know if you have any other questions or feedback. ๐Ÿ˜Š

View solution in original post

2 REPLIES 2

Kaniz_Fatma
Community Manager
Community Manager

Hi @bhawana, One possible solution is to use a caching layer in front of your database, such as Memcached or SafePeak. These tools can store the query results in a distributed memory cache and allow you to access them from multiple platforms. This way, you can avoid hitting the database directly for each query and reduce the load on your database server.

 

Another possible solution is to use a temporary table or a view in your database to store the query results. You can then join this table or view it with other tables or views in your queries and avoid repeating the same query. This way, you can leverage the indexes and statistics of your database to optimize your queries.

 

You can find more information about these solutions :

I hope this helps you with your problem. Please let me know if you have any other questions or feedback. ๐Ÿ˜Š

Kaniz_Fatma
Community Manager
Community Manager

I want to express my gratitude for your effort in selecting the most suitable solution. It's great to hear that your query has been successfully resolved. Thank you for your contribution.

 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group