cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Data_Analytics1
by Contributor III
  • 8143 Views
  • 8 replies
  • 2 kudos

TimeoutException: Futures timed out after [5 seconds]. I am getting this error while running few parallel jobs at an interval of 5 minutes.

java.util.concurrent.TimeoutException: Futures timed out after [5 seconds] at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:259) at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:263) at scala.concurrent.Await$.$...

  • 8143 Views
  • 8 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Mahesh Chahare​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...

  • 2 kudos
7 More Replies
arul_parthiban
by New Contributor
  • 418 Views
  • 0 replies
  • 0 kudos

How to customize the result of COPY INTO command?

COPY INTO statement produces results in following format, which is more similar to INSERT INTO statement results; also it's a summary of all the files loaded. Is there a way to customize in way that it produces the detailed results at file level?

image
  • 418 Views
  • 0 replies
  • 0 kudos
data_boy_2022
by New Contributor III
  • 1669 Views
  • 2 replies
  • 0 kudos

Resolved! Writing transformed DataFrame to a persistent table is unbearable slow

I want to transform a DF with a simple UDF. Afterwards I want to store the resulting DF in a new table (see code below)key = "test_key"   schema = StructType([ StructField("***", StringType(), True), StructField("yyy", StringType(), True), StructF...

  • 1669 Views
  • 2 replies
  • 0 kudos
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hello @Jan R​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 0 kudos
1 More Replies
nadia
by New Contributor II
  • 15255 Views
  • 2 replies
  • 2 kudos

Resolved! Executor heartbeat timed out

Hello, I'm trying to read a table that is located on Postgreqsl and contains 28 million rows. I have the following result:"SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in sta...

  • 15255 Views
  • 2 replies
  • 2 kudos
Latest Reply
jose_gonzalez
Moderator
  • 2 kudos

Hi @Boumaza nadia​ ,Did you check the executor 3 logs when the cluster was active? if you get this error message again, I will highly recommend to check the executor's logs to be sure on what was the cause of the issue.

  • 2 kudos
1 More Replies
Tahseen0354
by Contributor III
  • 17301 Views
  • 8 replies
  • 5 kudos

Resolved! Getting "Job aborted due to stage failure" SparkException when trying to download full result

I have generated a result using SQL. But whenever I try to download the full result (1 million rows), it is throwing SparkException. I can download the preview result but not the full result. Why ? What happens under the hood when I try to download ...

  • 17301 Views
  • 8 replies
  • 5 kudos
Latest Reply
rpshgupta
New Contributor III
  • 5 kudos

I am also having this issue again and again. I really want to understand what can we do to avoid this?

  • 5 kudos
7 More Replies
Labels