read percentage values in spark ( no casting )

sarvesh
Contributor III

I have a xlsx file which has a single column ;

percentage

30%

40%

50%

-10%

0.00%

0%

0.10%

110%

99.99%

99.98%

-99.99%

-99.98%

when i read this using Apache-Spark out put i get is,

|percentage|

+----------+

| 0.3|

| 0.4|

| 0.5|

| -0.1|

| 0.0|

| 0.0|

| 0.001|

| 1.1|

| 0.9999|

| 0.9998|

+----------+

expected output is ,

+----------+

|percentage|

+----------+

| 30%|

| 40%|

| 50%|

| -10%|

| 0.00%|

| 0%|

| 0.10%|

| 110%|

| 99.99%|

| 99.98%|

+----------+

My code -

val spark = SparkSession

.builder

.appName("trimTest")

.master("local[*]")

.getOrCreate()

val df = spark.read

.format("com.crealytics.spark.excel").

option("header", "true").

option("maxRowsInMemory", 1000).

option("inferSchema", "true").

load("data/percentage.xlsx")

df.printSchema()

df.show(10)

I Don't want to use casting or turning inferschema to false, i want a way to read percentage value as percentage not as double or string.