Prabakar
Databricks Employee
Databricks Employee

I believe this could be a known bug reported on the Elasticsearch Spark connector for Spark 3.0.

This connector is maintained by the Open source community and we don't have any ETA on the fix yet.

Bug details:

https://github.com/elastic/elasticsearch-hadoop/issues/1635

You can look for the latest connector to support Spark3.0 in Maven repo.

What is the DBR version that you are using for the cluster?