03-08-2022 07:15 PM
I expected the code below to print "hello" for each partition, and "world" for each record. But when I ran it the code ran but had no print outs of any kind. No errors either. What is happening here?
%scala
val rdd = spark.sparkContext.parallelize(Seq(1,2,3,4,5,6,7,8))
rdd.foreachPartition(partition => {
println("hello")
partition.foreach(fun=>{
println("world")
})
})
04-05-2022 05:24 PM
Hi @Leon P ,
You will need to check the executor's logs. Check the Stdout
03-08-2022 10:56 PM
Check your stdout logs
03-09-2022 02:49 AM
Is it lazy evaluated so you need to trigger action I guess
04-05-2022 05:24 PM
Hi @Leon P ,
You will need to check the executor's logs. Check the Stdout
04-25-2022 01:54 PM
Hi @Leon P ,
Just a friendly follow-up. Do you still need help or you were able to check the executor's logs and find the messages?
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.