https://community.databricks.com/t5/data-engineering/adding-a-dependent-library-to-a-job-task-permanently-adds-it-to/m-p/31911#M23263 <P>Why does adding a dependent library to a Job task also permanently add it to the entire cluster?</P><P></P><P>I am using python wheels, and even when I remove the dependent library from a Job task, the wheel is still part of the cluster configuration.</P><P></P><P>If I then update the version of&nbsp;wheel used by the task, I then shows multiple versions of the wheel in the cluster libraries. I have no way to know which one is being used.</P> Tue, 11 Jan 2022 01:07:28 GMT jpwp 2022-01-11T01:07:28Z https://community.databricks.com/t5/data-engineering/adding-a-dependent-library-to-a-job-task-permanently-adds-it-to/m-p/31911#M23263 <P>Why does adding a dependent library to a Job task also permanently add it to the entire cluster?</P><P></P><P>I am using python wheels, and even when I remove the dependent library from a Job task, the wheel is still part of the cluster configuration.</P><P></P><P>If I then update the version of&nbsp;wheel used by the task, I then shows multiple versions of the wheel in the cluster libraries. I have no way to know which one is being used.</P> Tue, 11 Jan 2022 01:07:28 GMT https://community.databricks.com/t5/data-engineering/adding-a-dependent-library-to-a-job-task-permanently-adds-it-to/m-p/31911#M23263 jpwp 2022-01-11T01:07:28Z