RuntimeError: It appears that you are attempting to reference SparkContext from a broadcast variable, action, or transformation. SparkContext can only be used on the driver, not in code that it run on workers. For more information, see SPARK-5063.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-07-2023
08:35 AM
- last edited on
03-20-2025
10:07 AM
by
Advika
I created the following model:
which calls get_identifier_information() which is as follows:
This is how I log the model
And this is the error I am running into:
RuntimeError: It appears that you are attempting to reference SparkContext from a broadcast variable, action, or transformation. SparkContext can only be used on the driver, not in code that it run on workers. For more information, see SPARK-5063.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-15-2023 11:01 PM
Hi @Nikhil Gajghate
Great to meet you, and thanks for your question!
Let's see if your peers in the community have an answer to your question. Thanks.

