how to upgrade pip associated with the default python

[ChriZhan_93142]

We have a job scheduled and submitted via Airflow to Databricks using api: api/2.0/jobs/runs/submit. Each time the job runs an ephemeral cluster will be launched and during the process a virtual env named: /local_disk0/.ephemeral_nfs/cluster_libraries/python (I believe this is the case for all clusters) is created and the job will run using this virtual env. Is there a way to upgrade the pip associated with this virtual env?

[Debayan]

Hi, I got an interesting article on the same. You can follow and let us know if this helps. 

Please tag @Debayan with your next comment which will notify me!