How to pass HBase-site.xml to a Databricks job?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-23-2022 01:12 AM
Hi, I have Azure Hbase cluster and Databricks. I want to run jobs on Databricks that write data to Hbase.
To connect to Hbase I need to get Hbase-site.xml and have it in the classpath or env of a job.
Question: How can I run the Databricks job with an attached Hbase-site.xml file?
Previously I have used a spark-submit command that allows passing additional configs. I cannot find any documented way to do it.
Is there any other way to read config directly from Hbase cluster?
Thanks.
- Labels:
-
AzureHbase
-
Databricks Job
-
Job
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-23-2022 12:07 PM
https://github.com/hortonworks-spark/shc - but not sure is it working with databricks
If you have a budget and want an easy solution, there is also JDBC driver https://www.cdata.com/kb/tech/hbase-jdbc-azure-databricks.rst
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-22-2022 05:57 PM
Hi @Vadim Z,
Just a friendly follow-up. Did the response from Hubert help you to resolve your issues? let us know if you still are looking for help

