Hi Community,

This might be a basic question, but I’m asking for educational purposes.

I noticed that in one of my jobs, the Spark UI shows -1 executors. Initially, I thought this might indicate that executors are idle, but that doesn’t seem to explain it fully. The job runs in a burst/triggered manner, so I would at least expect to see spikes in the Spark metrics view (e.g., Active Tasks). However, it shows a flat line at -1 for a long time, which doesn’t reflect the actual behavior.

Is this a known issue or possibly a bug?