Pylint github workflow

carlos_tasayco
Contributor

‎04-28-2025 03:32 PM

I am implementing this to my workflow however there is a problem I want to avoid 

Enrollment_Profile/mv_person_test_pylint.py:22:9: E0602: Undefined variable 'spark' (undefined-variable)

Can someone avoid that issue with spark dlt?

0 Kudos

samshifflett46
Databricks Partner

‎05-02-2025 12:03 PM

Hey @carlos_tasayco, it seems like you are running a test here. If you are running a test outside of a databricks environment (like in a CI pipeline) then you need to define a spark session manually. In the DBR (databricks runtime) the spark session is created automatically for you but when you run code locally or in a CI pipeline, then you need to build the spark session yourself and save it in the variable 'spark'. 

Check out this thread and look at MarkusFra's reply, he creates a Databricks Session object in his script which is what I think you need to do here.

https://community.databricks.com/t5/data-engineering/re-establish-sparksession-using-databricks-conn...

0 Kudos

carlos_tasayco
Contributor

‎05-02-2025 12:14 PM

Hi,

Yes I found that solution too, the problem after that was dbutils, pylint identify that as a no imported library. I found a solution at least for me in the pylintrc file I added this:

carlos_tasayco_0-1746213218337.png

Doing this I avoide this problem.

 

View solution in original post