Not able to run spark context by running python file in a cell using %sh

deficiant_codge
Contributor II

Hi Everyone,

I am trying to get spark context by running SparkSession builder command in a .py file and when i try to run it directl it works but if i got to a notebook and in a cell if i use %sh command to run python "myfilename" it gives an error that i am trying to create a new session. How to solve this?

Any help would be appreciated.

 

deficiant_codge_1-1689000063074.pngdeficiant_codge_2-1689000103143.png

 

 

Thanks