topic Re: Databricks Connector for Looker Studio – No Aggregation Pushdown + 1M Row Limit in Warehousing & Analytics

Databricks Connector for Looker Studio – No Aggregation Pushdown + 1M Row Limit

ahid — Sun, 13 Apr 2025 04:19:50 GMT

Hi Databricks Community,

I'm trying to understand which team is responsible for maintaining the Databricks Connector for Looker Studio . We’re currently facing a major performance bottleneck with how this connector operates.

Specifically:

The connector does not push down aggregate functions (e.g., SUM, AVG, COUNT) to Databricks.
Instead, it pulls all the raw data into Looker Studio and performs transformations there.
This is highly inefficient and quickly leads to hitting the 1 million row limit in Looker Studio.
Additionally, the lack of aggregation pushdown is severely impacting performance and usability for larger datasets.

We’re looking for:

Clarity on which team (Databricks, Google, or another) is responsible for the development and maintenance of this connector.
Any known workarounds, settings, or upcoming updates that support query pushdown or server-side aggregations.
Any alternative approaches to integrate Databricks with Looker Studio that bypass this limitation.

Appreciate any guidance, especially from those who’ve run into similar challenges or from the Databricks team.

Thanks in advance!

Re: Databricks Connector for Looker Studio – No Aggregation Pushdown + 1M Row Limit

Isi — Sun, 13 Apr 2025 14:57:26 GMT

Hi @ahid ,

Recently, I ran a small proof of concept comparing BI performance in Looker Studio using Databricks and Snowflake as backend engines.

I prefer not to share the specific results, but I can say this:

According to Databricks themselves:

“Our internal team has confirmed that the Looker connector is not our best-performing connector. There are known performance issues. Databricks has reported this to Google, but Google has declined to address it. We’re currently working to adapt to the situation on our side, but there’s no ETA for a fix at this time.”

From my perspective, this is a critical issue and should be treated as a priority by Databricks, especially given the growing number of users combining Databricks with Looker for enterprise BI reporting. However, at the moment, the response hasn’t been as proactive or urgent as it probably should be, and that’s directly impacting usability and trust in the integration. 😐

Regards,

Isi

Re: Databricks Connector for Looker Studio – No Aggregation Pushdown + 1M Row Limit

AndreyMirskiy — Wed, 16 Apr 2025 07:34:15 GMT

Thank you for the feedback!

Unfortunately there is a limitation in Looker Studio Community Connector API. getData method does not specify aggregation expectations for the data source. Therefore, a connector is expected to retrieve non-aggregated resultset.
I'd like to emphasize that it's not Databricks connector limitation. The same behavior can be observed with other community connectors, e.g. Snowflake connector.

We at Databricks are happy to improve the Databricks connector once Community Connector API provides relevant capabilities.