Pylint github workflow

[carlos_tasayco]

I am implementing this to my workflow however there is a problem I want to avoid 

Enrollment_Profile/mv_person_test_pylint.py:22:9: E0602: Undefined variable 'spark' (undefined-variable)

Can someone avoid that issue with spark dlt?

[samshifflett46]

Hey @carlos_tasayco, it seems like you are running a test here. If you are running a test outside of a databricks environment (like in a CI pipeline) then you need to define a spark session manually. In the DBR (databricks runtime) the spark session is created automatically for you but when you run code locally or in a CI pipeline, then you need to build the spark session yourself and save it in the variable 'spark'. 

Check out this thread and look at MarkusFra's reply, he creates a Databricks Session object in his script which is what I think you need to do here.

https://community.databricks.com/t5/data-engineering/re-establish-sparksession-using-databricks-conn...

[carlos_tasayco · Accepted solution]

Hi,

Yes I found that solution too, the problem after that was dbutils, pylint identify that as a no imported library. I found a solution at least for me in the pylintrc file I added this:

Doing this I avoide this problem.