The context is not shared between Scala and Python so you won't be able to access the same variables directly. However you can use createOrReplaceTempView to create a temporary view of your dataframe and read it in the other language with read_df = spark.sql("<sql query reading from view>").

Thanks @Tomasz Bacewicz​ ,

you mean, even if we use it in same notebook in databricks, it can't share among it, it maintain two different spark context for python and scala.

But I am getting another question like if we create TempView on dataframe, this tempView will maintain on which context level? Could you explain on it.

Thanks