topic Spark Error : RScript (1243) terminated unexpectedly: Cannot call r_RBufferinitialize(). in Data Engineering

Spark Error : RScript (1243) terminated unexpectedly: Cannot call r_RBufferinitialize().

chandan_a_v — Fri, 07 Jan 2022 13:21:17 GMT

grid_slice %>%

sdf_copy_to(

sc = sc,

name = "grid_slice",

overwrite = TRUE

) %>%

sdf_repartition(

partitions = min(n_executors * 3, NROW(grid_slice)),

partition_by = "variable"

) %>%

spark_apply(

f = slice_data_wrapper,

columns = c(

variable = "character",

max_slice = "integer",

n_slices = "integer"

context = list(

metadata = metadata,

s3_params = s3_params,

subfolder = "target_data/orig"

)

) %>%

compute() %>%

collect()

I got below attached issue when I tried to execute the above code. grid_slice is a tibble.

Tried different version of arrow (1.x, 4.x, 6.x) but didn't work.

ERROR sparklyr: RScript (1243) terminated unexpectedly: Cannot call r___RBuffer__initialize(). See https://arrow.apache.org/docs/r/articles/install.html for help installing Arrow C++ libraries.

Re: Spark Error : RScript (1243) terminated unexpectedly: Cannot call r_RBufferinitialize().

rajatha_d — Mon, 10 Jan 2022 08:10:40 GMT

Hi @Kaniz Fatma ,

I am also facing a similar issue, could you please provide the solution ASAP?

Thanks

Re: Spark Error : RScript (1243) terminated unexpectedly: Cannot call r_RBufferinitialize().

chandan_a_v — Mon, 10 Jan 2022 14:12:38 GMT

Hi @Kaniz Fatma

Did you find any solution? Please let us know

Re: Spark Error : RScript (1243) terminated unexpectedly: Cannot call r_RBufferinitialize().

chandan_a_v — Mon, 10 Jan 2022 14:31:59 GMT

Hi @Kaniz Fatma ,

Currently I am using R 3.6.2, I will upgrade it to 4.x and let you know. Thanks for the input.

Re: Spark Error : RScript (1243) terminated unexpectedly: Cannot call r_RBufferinitialize().

chandan_a_v — Tue, 11 Jan 2022 08:48:53 GMT

Hi @Kaniz Fatma ,

With R version 4.1.2, Spark version 2.4.5 and arrow version 5.0.0. The issue got fixed.

topic Spark Error : RScript (1243) terminated unexpectedly: Cannot call r___RBuffer__initialize(). in Data Engineering

Spark Error : RScript (1243) terminated unexpectedly: Cannot call r___RBuffer__initialize().

Re: Spark Error : RScript (1243) terminated unexpectedly: Cannot call r___RBuffer__initialize().

Re: Spark Error : RScript (1243) terminated unexpectedly: Cannot call r___RBuffer__initialize().

Re: Spark Error : RScript (1243) terminated unexpectedly: Cannot call r___RBuffer__initialize().

Re: Spark Error : RScript (1243) terminated unexpectedly: Cannot call r___RBuffer__initialize().

topic Spark Error : RScript (1243) terminated unexpectedly: Cannot call r_RBufferinitialize(). in Data Engineering

Spark Error : RScript (1243) terminated unexpectedly: Cannot call r_RBufferinitialize().

Re: Spark Error : RScript (1243) terminated unexpectedly: Cannot call r_RBufferinitialize().

Re: Spark Error : RScript (1243) terminated unexpectedly: Cannot call r_RBufferinitialize().

Re: Spark Error : RScript (1243) terminated unexpectedly: Cannot call r_RBufferinitialize().

Re: Spark Error : RScript (1243) terminated unexpectedly: Cannot call r_RBufferinitialize().