topic Re: RserveException: eval failed in Data Engineering

RserveException: eval failed

GC-James — Wed, 23 Feb 2022 12:27:28 GMT

Sometimes when I am running R code in a Databricks notebook I am given this error. The cell I am running fails, and my whole R 'session' seems to get screwed up. For example my stored variables disappear, and I have to re-load my packages etc. It is difficult to give a reproducible example because the behaviour is inconsistent. I can run the same code twice, one time it is fine, the next it is not. Below I give a picture of it happening in my notebook, and also the driver logs for the time it occurred. Grateful if someone has an idea as to what is causing this? Thanks.

Re: RserveException: eval failed

Atiq — Mon, 04 Apr 2022 08:50:47 GMT

Hey @Kaniz Fatma

We are also experiencing the same problem, and could not find a solution online. Could you kindly help us in this matter?

Re: RserveException: eval failed

GC-James — Mon, 04 Apr 2022 09:20:25 GMT

@Kaniz Fatma -- I'm not sure if the issue persists in RStudio or not. It would take quite some effort to port my code out of databricks and test.

Re: RserveException: eval failed

GC-James — Mon, 04 Apr 2022 09:21:35 GMT

@Atiquzzaman Ankur -- I think that this issue might be memory related. I think perhaps it happens when R tries to use more memory than is available to the Databricks cluster. Suggest monitoring the Ganglia UI just before you run a process that causes the crash. See if it 'tops out' the memory.

Re: RserveException: eval failed

Atiq — Mon, 04 Apr 2022 09:58:53 GMT

@James Smith Thank you for the suggestion :). I will try that out.

@Kaniz Fatma And from our end, the script works perfectly fine on RStudio. Only getting the issue in Databricks.

Re: RserveException: eval failed

GC-James — Mon, 04 Apr 2022 09:59:48 GMT

Well I have been limiting the memory usage of my code to 'solve' the problem so far. If it was the problem at all. I'm not sure.

Re: RserveException: eval failed

GC-James — Wed, 27 Apr 2022 09:47:54 GMT

Hello @Kaniz Fatma . None of the above replies have given a solution.

Re: RserveException: eval failed

data_warrior — Mon, 30 May 2022 16:44:01 GMT

Hello all @James Smith @Atiquzzaman Ankur @Kaniz Fatma ,

I am facing the exact same issue and do not know how to solve it. I tried freeing up some memory, But the problem persists. Please look at some of my screenshots.

It would be great if one of you could point me in the correct direction. I checked my memory consumption on driver node and I observed that it is maxing out for a short while. But I don't think this is the cause for failure as I have observed this multiple times before and the code still ran successfully without any error.

Best regards,

Akshay

Re: RserveException: eval failed

data_warrior — Mon, 30 May 2022 16:44:35 GMT

The error file is attached here.