Hey @carlos_tasayco, it seems like you are running a test here. If you are running a test outside of a databricks environment (like in a CI pipeline) then you need to define a spark session manually. In the DBR (databricks runtime) the spark session is created automatically for you but when you run code locally or in a CI pipeline, then you need to build the spark session yourself and save it in the variable 'spark'. 

Check out this thread and look at MarkusFra's reply, he creates a Databricks Session object in his script which is what I think you need to do here.

https://community.databricks.com/t5/data-engineering/re-establish-sparksession-using-databricks-conn...