topic Re: Exposing Data for Consumers in non-UC ADB in Data Engineering

Exposing Data for Consumers in non-UC ADB

fly_high_five — Fri, 05 Sep 2025 09:56:26 GMT

Hi,

I want to expose data to consumers from our non-UC ADB. Consumers would be consuming data mainly using SQL client like DBeaver. I tried SQL endpoint of Interactive Cluster and connected via DBeaver however when I try to fetch/export all rows of table it fails but succeds only for small number of rows upto 8k.

What is the best way to expose consumer layer?

Re: Exposing Data for Consumers in non-UC ADB

szymon_dybczak — Fri, 05 Sep 2025 10:10:10 GMT

Hi @fly_high_five ,

Which version of JDBC driver are you using? How you connection string looks like?

Re: Exposing Data for Consumers in non-UC ADB

fly_high_five — Fri, 05 Sep 2025 10:38:24 GMT

Hi @szymon_dybczak

I am using latest JDBC driver 2.7.3 https://www.databricks.com/spark/jdbc-drivers-archive

And my JDBC url comes from JDBC endpoint of Interactive Cluster.

jdbc:databricks://adb-{workspace_id}.azuredatabricks.net:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/{workspace_id}/{cluster_id};AuthMech=11;Auth_Flow=2;TokenCachePassPhrase=U2MToken;EnableTokenCache=0

Re: Exposing Data for Consumers in non-UC ADB

szymon_dybczak — Fri, 05 Sep 2025 11:24:07 GMT

Looking at the error I noticed the weird name of the storage account. So now I'm thinking that your jdbc connection is using CloudFetch.

"Cloud Fetch, a capability that fetches query results through the cloud storage that is set up in your Azure Databricks deployment.

Query results are uploaded to an internal DBFS storage location as Arrow-serialized files of up to 20 MB. When the driver sends fetch requests after query completion, Azure Databricks generates and returns shared access signatures to the uploaded files. The JDBC driver then uses the URLs to download the results directly from DBFS."

Can you add following option to your JDBC url? It will disable cloudFetch. Once you set this option try again.

EnableQueryResultDownload=0

Re: Exposing Data for Consumers in non-UC ADB

fly_high_five — Fri, 05 Sep 2025 14:24:48 GMT

Hi @szymon_dybczak ,
Your recommendation worked. I modified JDBC url to

I tried another solution which looks essentially same what @szymon_dybczak you recommended. Solution is attached.