topic Re: Installing new libraries on Azure Databricks Clusters in Data Engineering

Installing new libraries on Azure Databricks Clusters

Nuthan_1994 — Wed, 13 Oct 2021 21:28:47 GMT

Hi Everyone,

I was trying to install the newest python version on the Databricks Clusters and it has the runtime version 7.3 LTS, but no matter how many times I try it keeps installing the 3.7.5 version of python.

I know that Runtime version 7.3 LTS comes with Python 3.7.5 but I was wondering if I could install a newer version of Python? Is it possible to do that?

Re: Installing new libraries on Azure Databricks Clusters

dazfuller — Thu, 14 Oct 2021 08:31:14 GMT

I've done this before using a custom docker image, but even then the runtime itself continues to use the version of python 3 which is installed as part of the OS. The easiest way to get to a newer version is to use a newer runtime. If you're sticking with LTS releases then you could jump to the 9.1 LTS release which uses Python 3.8.8

Re: Installing new libraries on Azure Databricks Clusters

Nuthan_1994 — Thu, 14 Oct 2021 12:27:40 GMT

Alright, thanks!

Re: Installing new libraries on Azure Databricks Clusters

Prabakar — Sun, 28 Nov 2021 22:11:28 GMT

Hi @Nuthan Peddapurapu , I could see the question was answered and if you find the answer to be helpful and resolved your query, would you be happy to mark it as best so others can quickly find the solution in the future.