cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

naveenprabhun
by New Contributor III
  • 2283 Views
  • 2 replies
  • 3 kudos

Resolved! Unable to read data from ElasticSearch using Databricks (AWS) Cannot detect ES version - Caused by: org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [IP:PORT]

I am trying to read data from ElasticSearch(ES Version 8.5.2) using PySpark on Databricks (13.0 (includes Apache Spark 3.4.0, Scala 2.12)). The ecosystem is on AWS.I am able to run a curl command on the Databricks notebook to the ES ip:port and fetch...

ErrorScreenshot Screenshot 2023-06-01 at 1.25.29 PM
  • 2283 Views
  • 2 replies
  • 3 kudos
Latest Reply
Hoviedo
New Contributor II
  • 3 kudos

I have the same problem, did you find any solution? thanks

  • 3 kudos
1 More Replies
Raymond_Garcia
by Contributor II
  • 800 Views
  • 0 replies
  • 1 kudos

EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[elasticsearch_server:80]]

Hi, I wondered if some of you have had this issue before and how it can be solved. In a Databricks Job, we have a UBQ with a Painless script for ES. these are the options. Staging and prod are the same configurations, but Staging is failing with the ...

  • 800 Views
  • 0 replies
  • 1 kudos
Raymond_Hu
by New Contributor
  • 12403 Views
  • 1 replies
  • 0 kudos

ConnectException error

I'm using PySpark on Databricks and trying to pivot a 27753444 X 3 matrix. If I do it in Spark DataFrame: df = df.groupBy("A").pivot("B").avg("C") it takes forever (after 2 hours and I canceled it). If I convert it to pandas dataframe and then pivo...

  • 12403 Views
  • 1 replies
  • 0 kudos
Latest Reply
shyam_9
Valued Contributor
  • 0 kudos

Hi @Raymond_Hu,This means that the driver crashed because of an OOM (Out of memory) exception and after that, it's not able to establish a new connection with the driver. Please try below optionsTry increasing driver-side memory and then retry.You ca...

  • 0 kudos
Labels