sparkR.session

[Flachboard84]

Why might this be erroring out? My understanding is that SparkR is built into Databricks.

Code:

library(SparkR, include.only=c('read.parquet', 'collect'))

sparkR.session()

Error:

Error in sparkR.session(): could not find function "sparkR.session"

[shan_chandra]

@Flachboard84  - could you please try the below code ?

library(SparkR, include.only=c('read.parquet', 'collect'))

sparkR.session() is not required on the databricks notebook. This should be handled automatically.

Reference: https://docs.databricks.com/en/sparkr/overview.html

[Flachboard84]

Thanks for the reply!

Now I'm getting this:

Warning in sparkR.callJMethod(SparkR:::getSparkContext(), "setLocalProperty", : could not find function "sparkR.callJMethod"

[shan_chandra]

@Flachboard84  - could you please share the associated code?

[Flachboard84]

It happens with any code; even something as simple as...

x <- 2 + 2