saurabh18cs
Honored Contributor III

Hi @ajay_wavicle 

The Spark session never disappears %sh simply runs outside it.

do this:

from mymodule import myfunc
myfunc(spark)

dont do this :


%sh
python my_script.py