cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Unable to access Scala and python variables in-between shells in same notebook.

Data_Engineer3
Contributor III

I am facing issue in while accessing python data frame in Scala shell and vice versa. I am getting error variable not defined.

2 REPLIES 2

tomasz
Databricks Employee
Databricks Employee

The context is not shared between Scala and Python so you won't be able to access the same variables directly. However you can use createOrReplaceTempView to create a temporary view of your dataframe and read it in the other language with read_df = spark.sql("<sql query reading from view>").

Thanks @Tomasz Bacewiczโ€‹ ,

you mean, even if we use it in same notebook in databricks, it can't share among it, it maintain two different spark context for python and scala.

But I am getting another question like if we create TempView on dataframe, this tempView will maintain on which context level? Could you explain on it.

Thanks

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group