topic Re: How to pass HBase-site.xml to a Databricks job? in Data Engineering

How to pass HBase-site.xml to a Databricks job?

Vadim1 — Thu, 23 Jun 2022 08:12:03 GMT

Hi, I have Azure Hbase cluster and Databricks. I want to run jobs on Databricks that write data to Hbase.

To connect to Hbase I need to get Hbase-site.xml and have it in the classpath or env of a job.

Question: How can I run the Databricks job with an attached Hbase-site.xml file?

Previously I have used a spark-submit command that allows passing additional configs. I cannot find any documented way to do it.

Is there any other way to read config directly from Hbase cluster?

Thanks.

Hubert-Dudek — Thu, 23 Jun 2022 19:07:56 GMT

https://github.com/hortonworks-spark/shc - but not sure is it working with databricks

If you have a budget and want an easy solution, there is also JDBC driver https://www.cdata.com/kb/tech/hbase-jdbc-azure-databricks.rst

jose_gonzalez — Tue, 23 Aug 2022 00:57:46 GMT

Hi @Vadim Z,

Just a friendly follow-up. Did the response from Hubert help you to resolve your issues? let us know if you still are looking for help