Databricks SQL not displaying all the databases that i have on my cluster.

[DB_007]

I have a cluster running on 7.3 LTS and it has about 35+ databases. When i tried to setup an endpoint on Databricks SQL, i do not see any database listed.

[Atanu]

Could you please share a screenshot?

[DB_007]

Attached pic

[Atanu]

@Arif Ali​  Not seeing the pic. Could you please upload again.

[Atanu]

Another question @Arif Ali​  what is the permission level you inherit from this cluster?

[DB_007]

I have admin privileges on the cluster and also full access to the endpoint. Metastore is external and do i need to do anything in order to get databases listed on the Databricks SQL.

[Anonymous]

@Arif Ali​ There might be a couple of reasons.

• Table ACL is enabled by default in DB SQL, so please ensure that you have permission to the underlying databases as mentioned in https://docs.databricks.com/sql/user/security/data-access-overview.html

1. If you're using an external metastore, please add the required Data access configurations as mentioned in https://docs.databricks.com/sql/admin/data-access-configuration.html

[User16871418122 · Accepted solution]

hi @Arif Ali​  You may have to check the data access config to add the params for external metastore:

spark.hadoop.javax.jdo.option.ConnectionDriverName org.mariadb.jdbc.Driver

spark.hadoop.javax.jdo.option.ConnectionUserName <mysql-username>

spark.hadoop.javax.jdo.option.ConnectionPassword <mysql-password>

spark.sql.hive.metastore.version <hive-version>

spark.sql.hive.metastore.jars <hive-jar-source>

 

If the data access config is already added, it is likely the metastore initialization is taking longer. ie( spark to apply the spark configs to load metastore). If given some more time it will initialize the metastore and enables schema browsing. If still issues, the configs may be wrong, you shall have them validated on interactive Databricks cluster before applying in SQL endpoints.

In the near future with the Unity catalog, the schema browsing is cluster less. The dependency on having a cluster is removed. 😊