I am trying to create a SQL Dashboard on top of a streaming dataset (Delta format). I created multiple queries referencing the file on the datalake, not a hive table. With these queries I created multiple visualizations in a Databricks SQL Dashboard. I added an automated refresh, as the new data in the streaming dataset needs to be displayed as fast as possible. This should refresh the data every 1 minute (More real time would be nice, but 1 minute works).
I would like to keep the dashboard open, so that I always have the visuals showing the data that has been refreshed less than one minute ago. However, the refresh keeps loading. The screenshot below shows that there is a refresh symbol going on at every visual, but the data is not being refreshed. In the screenshot it seems that the refresh is already taking more than 7 minutes.
If I click manually on the refresh button on the top right corner of the dashboard, the refresh takes never more than 10 seconds.
My question is, is this expected behaviour? Do we need to refresh the browser page before we can see the updates of the Dashboard Refresh? Am I doing something wrong? Or is this just a bug?
@Jasper Puts :
Here are some potential solutions or next steps you can try:
@Suteja Kanuri Thanks for the response.
-There is no scheduled refresh on the queries. Is this something that is necessary? I assumed that refreshing the dashboard automatically will also refresh the query?
-I think the problem is linked to answer 2. It appears that the dataset keeps refreshing, but when I actually refresh the page, the dashboard is also refreshed. The goal is to display the dashboard somewhere on a tv/monitor. It would miss its purpose if a manual refresh needs to be done every time. Any ideas how this can be fixed? I tried it on both Chrome and Firefox.
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!