Can you share variables defined in a Python based cell with Scala cells?

[aladda · Accepted solution]

Each databricks notebook has an indepedent Scala and Python shell/REPL. Thus variables defined in one can't be shared with the other. However Temp-Views are kept within the Notebook's own spark session and thus can be shared across different language cells and is a common technique to do so

[Imtiyaz_Shaikh]

The workaround is available here.

Idea is to use spark.conf.set(), spark.conf.get() methods.