We have an intermittant issue where occasionally a partition in our Power BI Import Dataset times out at 5 hours. When I look at Query History in Databricks SQL, I see a query that failed with the following error message: "Query has been timed out due to inactivity". That query info screen, in one example, shows that the total Wall-clock duration was 15.587seconds. But the Result fetching by client row shows 1.04hours. Power BI doesn't seem to know that the query failed as it continues to wait for results until it times out at 5 hours (it is a Power BI Premium Per User dataset). In other examples, the query wall-clock sometimes runs for a few minutes, but the Result fetching by client row always shows a very long time (over an hour to a few hours) relative to the wall-clock.
I would like to know what this error message means in Databricks. It looks like the query is returning rows in a normal amount of time, but the query still fails. I have not been able to find what this error message indicates I should address.
I have attached a screenshot of the example query I mentioned above that failed with this error.
This error is intermittent as the model does not usually time out, but when it does, I always find a query with this error message in query history during the time of the load. It is also not always on the same partition, but it is always on a partition on one of the larger tables. The full refresh on this particular data model usually takes around 2 hours. When this error occurs, it times out at 5 hours. Then the retry will complete in the normal 2 hours.
Another issue, albeit this issue is likely with Power BI and not Databricks, is that the query fails with this inactivity message long before the Dataset times out at 5 hours. In this example, the query failed 2 hours before the Power BI Dataset timed out due to inactivity, so Power BI does not know the query failed and continues to wait for results until the refresh times out at the Premium timeout of 5 hours.
Thank you in advance for any information you are able to provide that may help me narrow down how to address this.