cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

User15986662700
by New Contributor III
  • 2927 Views
  • 4 replies
  • 1 kudos
  • 2927 Views
  • 4 replies
  • 1 kudos
Latest Reply
User15986662700
New Contributor III
  • 1 kudos

Yes, it is possible to connect databricks to a kerberized hbase cluster. The attached article explains the steps. It consists of setting up a kerberos client using a keytab in the cluster nodes, installing the hbase-spark integration library, and set...

  • 1 kudos
3 More Replies
Sweetnesh
by New Contributor
  • 1078 Views
  • 2 replies
  • 0 kudos

Not able to read S3 object through AssumedRoleCredentialProvider

SparkSession spark = SparkSession.builder() .appName("SparkS3Example") .master("local[1]") .getOrCreate(); spark.sparkContext().hadoopConfiguration().set("fs.s3a.access.key", S3_ACCOUNT_KEY); spark.sparkContext().hadoopConf...

  • 1078 Views
  • 2 replies
  • 0 kudos
Latest Reply
Vartika
Moderator
  • 0 kudos

Hi @Sweetnesh Dholariya​,Does @Debayan Mukherjee​'s response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?Thanks!

  • 0 kudos
1 More Replies
cconnell
by Contributor II
  • 444 Views
  • 1 replies
  • 2 kudos

www.linkedin.com

Here is an article I wrote that puts Databricks in a historical context (why was it developed?) and provides introductory steps to help a newbie get started. Feel free to copy/link as you want.https://www.linkedin.com/pulse/databricks-introduction-ch...

  • 444 Views
  • 1 replies
  • 2 kudos
Latest Reply
Kaniz
Community Manager
  • 2 kudos

Hi @Chuck Connell​ , Thank you for posting such an informative article!

  • 2 kudos
ZeykUtra
by New Contributor
  • 545 Views
  • 0 replies
  • 0 kudos

java.io.IOException: While processing file s3://test/abc/request_dt=2021-07-28/someParquetFile. [XYZ] BINARY is not in the store

Hi Team, I am facing an issue "java.io.IOException: While processing file s3://test/abc/request_dt=2021-07-28/someParquetFile. [XYZ] BINARY is not in the store" The things i did before getting the above exception: 1. Alter table tableName1 add colum...

  • 545 Views
  • 0 replies
  • 0 kudos
User16869510359
by Esteemed Contributor
  • 547 Views
  • 1 replies
  • 0 kudos

Resolved! Does Table ACL support column-level security like Ranger?

I have used Ranger in Apache Hadoop and it works fine for my use case. Now that I am migrating my workloads from Apache Hadoop to Databricks

  • 547 Views
  • 1 replies
  • 0 kudos
Latest Reply
User16869510359
Esteemed Contributor
  • 0 kudos

Currently, Table ACL does not support column-level security. There are several tools like Privcera which has better integration with Databricks.

  • 0 kudos
Mallesh
by New Contributor
  • 9244 Views
  • 1 replies
  • 0 kudos

How can i read parquet file compressed by snappy?

Hi All, I wanted to read parqet file compressed by snappy into Spark RDD input file name is: part-m-00000.snappy.parquet i have used sqlContext.setConf("spark.sql.parquet.compression.codec.", "snappy") val inputRDD=sqlContext.parqetFile(args(0)) whe...

  • 9244 Views
  • 1 replies
  • 0 kudos
Latest Reply
raela
New Contributor III
  • 0 kudos

Have you tried sqlContext.read.parquet("/filePath/") ?

  • 0 kudos
Labels