cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Connect from Databricks to Hbase HDinsight cluster.

Vadim1
New Contributor III

Hi, I have Databricks installation in Azure. I want to run a job that connects to HBase in a separate HDinsight cluster.

What I tried:

  1. Created a peering between base cluster and Databricks vNets.
  2. I can ping IPs of Hbase zookeeper nodes but I cannot access them by domain name (zk1.abc.net, etc). That's why I'm getting timeout and host-not-found errors.

As I understand I need to set up a DNS to be able to communicate with Hbase but I cannot change Databricks DNS (locked by Databricks security policy).

Can you help me with this, please?

Maybe you can refer to the guide that I can use?

Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions

Vadim1
New Contributor III

Thanks for the response, werners. I found that my Databricks cluster was created with a private virtual network that cannot be changed, so I deployed a new Databricks cluster with my own virtual network and this resolved all issues with connectivity.

View solution in original post

3 REPLIES 3

-werners-
Esteemed Contributor III

@Vadim Z​, Could it be a permission setting for your user?

Because you can change DNS settings on Databricks (f.e. https://kb.databricks.com/cloud/custom-dns-routing.html)

Vadim1
New Contributor III

Thanks for the response, werners. I found that my Databricks cluster was created with a private virtual network that cannot be changed, so I deployed a new Databricks cluster with my own virtual network and this resolved all issues with connectivity.

User16764241763
Honored Contributor

Vadim,

Thank you for the response. Appreciate it.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.