Why does adding a dependent library to a Job task also permanently add it to the entire cluster?

I am using python wheels, and even when I remove the dependent library from a Job task, the wheel is still part of the cluster configuration.

If I then update the version of wheel used by the task, I then shows multiple versions of the wheel in the cluster libraries. I have no way to know which one is being used.