sparkR.session

Flachboard84
New Contributor II

Why might this be erroring out? My understanding is that SparkR is built into Databricks.

Code:

library(SparkR, include.only=c('read.parquet', 'collect'))
sparkR.session()
 
Error:
Error in sparkR.session(): could not find function "sparkR.session"

shan_chandra
Databricks Employee
Databricks Employee

@Flachboard84  - could you please try the below code ?

library(SparkR, include.only=c('read.parquet', 'collect'))

sparkR.session() is not required on the databricks notebook. This should be handled automatically.

Reference: https://docs.databricks.com/en/sparkr/overview.html

Thanks for the reply!

Now I'm getting this:

Warning in sparkR.callJMethod(SparkR:::getSparkContext(), "setLocalProperty", : could not find function "sparkR.callJMethod"

@Flachboard84  - could you please share the associated code?

 

Flachboard84
New Contributor II

It happens with any code; even something as simple as...

x <- 2 + 2