Hi @Aman Sehgal​ 

I am trying to read elastic data by directly connect to it.

I am using below snippet

df = spark.read.format("org.elasticsearch.spark.sql")

 .option("es.read.metadata", "false")

 .option("spark.es.nodes.discovery", "true")

 .option("es.net.ssl", "false")

 .option("es.index.auto.create", "true")

 .option("es.field.read.empty.as.null", "no")

 .option("es.read.field.as.array.exclude","true")

 .option("spark.serializer", "org.apache.spark.serializer.KryoSerializer")

 .option("es.nodes", "*")

 .option("es.nodes.wan.only", "true")

 .option("es.net.http.auth.user", elasticUsername)

 .option("es.net.http.auth.pass", elasticPassword)

 .option("es.resource", "indexname")

But I am getting runtime error showing that

RuntimeException: Error while encoding: java.lang.RuntimeException: scala.collection.convert.Wrappers$JListWrapper is not a valid external type for schema of string

Caused by: RuntimeException: scala.collection.convert.Wrappers$JListWrapper is not a valid external type for schema of string

do you have solution to it?

Note: i think error due to schema getting generated by spark is not matching with schema present in elastic.

Thanks