https://community.databricks.com/t5/get-started-discussions/how-to-upgrade-pip-associated-with-the-default-python/m-p/48982#M1536 <P>We have a job scheduled and submitted via Airflow to Databricks using api:&nbsp;<SPAN>api/2.0/jobs/runs/submit. Each time the job runs an ephemeral cluster will be launched and during the process a virtual env named:&nbsp;/local_disk0/.ephemeral_nfs/cluster_libraries/python (I believe this is the case for all clusters) is created and the job will run using this virtual env. Is there a way to upgrade the pip associated with this virtual env?</SPAN></P> Wed, 11 Oct 2023 20:33:31 GMT ChriZhan_93142 2023-10-11T20:33:31Z https://community.databricks.com/t5/get-started-discussions/how-to-upgrade-pip-associated-with-the-default-python/m-p/48982#M1536 <P>We have a job scheduled and submitted via Airflow to Databricks using api:&nbsp;<SPAN>api/2.0/jobs/runs/submit. Each time the job runs an ephemeral cluster will be launched and during the process a virtual env named:&nbsp;/local_disk0/.ephemeral_nfs/cluster_libraries/python (I believe this is the case for all clusters) is created and the job will run using this virtual env. Is there a way to upgrade the pip associated with this virtual env?</SPAN></P> Wed, 11 Oct 2023 20:33:31 GMT https://community.databricks.com/t5/get-started-discussions/how-to-upgrade-pip-associated-with-the-default-python/m-p/48982#M1536 ChriZhan_93142 2023-10-11T20:33:31Z https://community.databricks.com/t5/get-started-discussions/how-to-upgrade-pip-associated-with-the-default-python/m-p/48998#M1540 <P>Hi, I got an interesting article on the same. You can&nbsp;<A href="https://www.freecodecamp.org/news/pip-upgrade-and-how-to-update-pip-and-python/" target="_self">follow</A>&nbsp;and let us know if this helps.&nbsp;</P> <P><SPAN>Please tag&nbsp;</SPAN><A href="https://community.databricks.com/t5/user/viewprofilepage/user-id/26078" target="_blank">@Debayan</A><SPAN>&nbsp;with your next comment which will notify me!</SPAN></P> Thu, 12 Oct 2023 06:39:54 GMT https://community.databricks.com/t5/get-started-discussions/how-to-upgrade-pip-associated-with-the-default-python/m-p/48998#M1540 Debayan 2023-10-12T06:39:54Z