Reading information_schema tables through JDBC connection

Immassive — Tue, 07 Nov 2023 11:50:47 GMT

Hi,

I am using Unity Catalog as storage for data. I have an external system that establishes connection to Unity Catalog via a JDBC connection using the Databricks driver:

Configure the Databricks ODBC and JDBC drivers - Azure Databricks | Microsoft Learn

However, I am only able to list Delta Tables through the JDBC connection. I am interested in reading the tables from system.information_schema to use some of the tables' metadata. For example, I am using SAS in my external system and my code looks like the following:

%let MYDBRICKS=<my-db-url>; %let MYHTTPPATH=<cluster-http-path>; %let CATALOG=system; %let SCHEMA=information_schema; libname dbricks jdbc driverclass="com.databricks.client.jdbc.Driver" url="jdbc:databricks://&MYDBRICKS:443/&SCHEMA;transportMode=http;ssl=1;httpPath=&MYHTTPPATH;AuthMech=11;Auth_Flow=0;Auth_AccessToken=&AZTOKEN;LogLevel=5;QueryPassthrough=true;UseNativeQuery=1;StripCatalogName=0;ConnCatalog=&CATALOG";

So, the connection works, and I can see Delta Tables, but no normal SQL tables.

The authentication is based on a Azure token (Microsoft Entra ID), however, I've tested with a Personal Access Token as well, and has no effect.

Is this just a limitation of the JDBC driver itself or is it somehow possible to configure it to also be able to read normal sql tables and not just Delta Tables?

Re: Reading information_schema tables through JDBC connection

Immassive — Wed, 08 Nov 2023 11:56:40 GMT

Note: I can see the tables of the system.information schema in the UI of Databricks and read them there.

topic Reading information_schema tables through JDBC connection in Data Engineering

Reading information_schema tables through JDBC connection

Re: Reading information_schema tables through JDBC connection