06-25-2021 12:54 PM
The below code gives different result when executed using DB Connect and a Notebook
sc = spark.sparkContext a = sc.accumulator(0) rdd = sc.parallelize([1, 2, 3]) def f(x): global a a.add(x) rdd.foreach(f) rdd.count() print(a.value)
06-25-2021 12:55 PM
This is a known limitation that accumulators do not work with DB Connect.
View solution in original post
never-displayed
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.