cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
User15986662700
by • New Contributor III
  • 4188 Views
  • 4 replies
  • 1 kudos

How to integrate Databricks and Spark to Secure HBase cluster with Kerberos?

Data Engineering
Reply
  • 4188 Views
  • 4 replies
  • 1 kudos
View Replies
Latest Reply
User15986662700
New Contributor III
  • 1 kudos

Yes, it is possible to connect databricks to a kerberized hbase cluster. The attached article explains the steps. It consists of setting up a kerberos client using a keytab in the cluster nodes, installing the hbase-spark integration library, and set...

  • 1 kudos
3 More Replies
mshettar
by • New Contributor II
  • 1619 Views
  • 2 replies
  • 2 kudos

newAPIHadoopRDD Spark API doesn't retrieve unflushed data written to Hbase table

Reading from an HBase table with a few hundred records that haven't been persisted (flushed) to HDFS doesn't show up in Spark. However, the records become visible after forced flush via Hbase shell or system triggered flush (when size of Memstore cro...

Data Engineering
Reply
  • 1619 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@Manjunath Shettar​ :It seems that the issue is related to the fact that the records in the HBase table have not been flushed to HDFS and are still stored in the Memstore. Spark's newAPIHadoopRDD API reads data from the HBase table through HBase's Ta...

  • 2 kudos
1 More Replies
Mahesha999
by • New Contributor II
  • 4781 Views
  • 3 replies
  • 0 kudos

Resolving NoClassDefFoundError: org/apache/spark/Logging exception

I was trying out hbase-spark connector. To start with, I am trying out this code. My pom dependencies are: <dependencies> <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-core_2.11</artifactId> <version...

Data Engineering
Reply
  • 4781 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
User16301467518
New Contributor II
  • 0 kudos

The alpha of hbase-spark you're using depends on Spark 1.6 -- see hbase-spark/pom.xml:L33 -- so you'll probably have to stick with 1.6 if you want to use that published jar. For reasons I don't understand hbase-spark was removed in the last couple o...

  • 0 kudos
2 More Replies
Labels