How to read a fixed length file in Spark using DataFrame API and SCALA

alexraj84
New Contributor

‎08-04-2016 10:52 AM

I have a fixed length file ( a sample is shown below) and I want to read this file using DataFrames API in Spark using SCALA(not python or java). Using DataFrames API there are ways to read textFile, json file and so on but not sure if there is a way to read a fixed-length file. I was searching the internet for this and found a github link, but I got to download 

spark-fixedwidth-assembly-1.0.jar
for this purpose however I was unable to figure out the jar anywhere. I am completely lost here and need your suggestions and help.

Here is the file 

    56 apple     TRUE 0.56
    45 pear      FALSE1.34
    34 raspberry TRUE 2.43
    34 plum      TRUE 1.31
    53 cherry    TRUE 1.4 
    23 orange    FALSE2.34
    56 persimmon FALSE23.2

The fixed width of each columns are 3, 10, 5, 4

Please suggest your opinion.

Labels:
0 Kudos