cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

How can i read parquet file compressed by snappy?

Mallesh
New Contributor

Hi All,

I wanted to read parqet file compressed by snappy into Spark RDD

input file name is: part-m-00000.snappy.parquet

i have used sqlContext.setConf("spark.sql.parquet.compression.codec.", "snappy")

val inputRDD=sqlContext.parqetFile(args(0))

whenever im trying to run im facing java.lang.IlligelArgumentException : Illegel character in opaque part at index 2

i tried renaming the input file like input_data_snappy.parquet,then also im getting same exception.

please help me for this

Thanks for advance.

1 REPLY 1

raela
New Contributor III
New Contributor III

Have you tried sqlContext.read.parquet("/filePath/") ?

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.