Unable to access Scala and python variables in-between shells in same notebook.

[Data_Engineer3]

I am facing issue in while accessing python data frame in Scala shell and vice versa. I am getting error variable not defined.

[tomasz]

The context is not shared between Scala and Python so you won't be able to access the same variables directly. However you can use createOrReplaceTempView to create a temporary view of your dataframe and read it in the other language with read_df = spark.sql("<sql query reading from view>").

[Data_Engineer3]

Thanks @Tomasz Bacewicz​ ,

you mean, even if we use it in same notebook in databricks, it can't share among it, it maintain two different spark context for python and scala.

But I am getting another question like if we create TempView on dataframe, this tempView will maintain on which context level? Could you explain on it.

Thanks