Variables are not shared between language contexts.

Ugly workaround: you could do something like this to pass your python variable to the spark context:

%python
d1 = {1: "a", 2:"b", 3:"c"}
spark.conf.set('d1', str(d1))

%scala
println( spark.conf.get("d1") )

However, you lose variable type. You only can pass on strings.