ODBC Connection Does Not Disconnect

[bradleyjamrozik]

I have an on-premises Power BI Report Server that uses the Simba Spark ODBC Driver (2.8) to connect to Databricks. It can connect to a serverless warehouse successfully and run its queries, but it never seems to disconnect the session, and so the warehouse never registers as "idle" and auto-terminates. Has anyone seen this before? Any suggestions/recommendations? Thanks.

[bradleyjamrozik]

Enabling "Driver Config Take Precedence" seems to be doing something. I'm going to monitor and report back.

[bradleyjamrozik]

It's working sometimes. Only correlation I have found so far is that a successful query will disconnect as expected but any error will keep the connection to the warehouse open indefinitely. 

[gmiguel]

Hi @bradleyjamrozik ,

The problem is that the current designed behavior of Power BI can lead to the connection not being closed under certain conditions.

In short, since Databricks is Async, if an error occurs while fetching data, the connection may not be closed gracefully. This causes Databricks Sql Serverless to not terminate after the idle timeout is reached, even without any queries running.

Microsoft has a backlog item to improve their process of closing connections on errors. They said it is a complex change and needs to be prioritized. As of now they have no ETA confirmed by the Product Team.

This was the important part of the response I received from the Microsoft PowerBI Technical Support Team.

Until someone fixes it, Sql Serverless may keep active 24x7, for something that should be kept active when needed.