RuntimeError: It appears that you are attempting to reference SparkContext from a broadcast variable, action, or transformation. SparkContext can only be used on the driver, not in code that it run on workers. For more information, see SPARK-5063.

[Nikhil3107]

I created the following model:

 

which calls get_identifier_information() which is as follows:

 

This is how I log the model

 

And this is the error I am running into:

RuntimeError: It appears that you are attempting to reference SparkContext from a broadcast variable, action, or transformation. SparkContext can only be used on the driver, not in code that it run on workers. For more information, see SPARK-5063.

[Anonymous]

Hi @Nikhil Gajghate​ 

Great to meet you, and thanks for your question!

Let's see if your peers in the community have an answer to your question. Thanks.