cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
pepe
by • New Contributor II
  • 8756 Views
  • 2 replies
  • 1 kudos

Why can't I install python libraries when i update cluster runtime from 10.1 to 12.1?

This same question was asked here 9 months ago without any answer:https://community.databricks.com/s/question/0D58Y000096VjKrSAK/managedlibraryinstallfailed-when-changing-databricks-runtime-version-from-91-to-110I was using runtime 9.1, and then upgr...

Data Engineering
Reply
  • 8756 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @JOSE RODRIGUEZ​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...

  • 1 kudos
1 More Replies
tatekeller
by • New Contributor
  • 1764 Views
  • 1 replies
  • 0 kudos

Can you access a repo file in an init script?

I'd like to configure a cluster with python libraries as defined in a requirements file. I have a pip requirements.txt file in a private repo which I have integrated on Databricks (and I can access it through the UI and view it on Databricks). I upda...

Data Engineering
Reply
  • 1764 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
sher
Valued Contributor II
  • 0 kudos

you can install in a cluster

  • 0 kudos
spott_submittab
by • New Contributor II
  • 5989 Views
  • 7 replies
  • 9 kudos

How are people developing python libraries for use within a team on databricks?

In the past, before databricks, I would try and pull commonly used functions and features out of notebooks and save them in a python library that the whole team would work on and develop. This allowed for good code reuse and maintaining best practic...

Data Engineering
Reply
  • 5989 Views
  • 7 replies
  • 9 kudos
View Replies
Latest Reply
dazfuller
Contributor III
  • 9 kudos

The way we do this is to package as much re-usable code up into a common library as possible and then test it to within an inch of it's life with unit tests (I tend to use unittest for lower barrier to entry, but which ever framework works best for y...

  • 9 kudos
6 More Replies
Nuthan_1994
by • New Contributor II
  • 3811 Views
  • 3 replies
  • 3 kudos

Resolved! Installing new libraries on Azure Databricks Clusters

Hi Everyone,I was trying to install the newest python version on the Databricks Clusters and it has the runtime version 7.3 LTS, but no matter how many times I try it keeps installing the 3.7.5 version of python.I know that Runtime version 7.3 LTS co...

Data Engineering
Reply
  • 3811 Views
  • 3 replies
  • 3 kudos
View Replies
Latest Reply
dazfuller
Contributor III
  • 3 kudos

I've done this before using a custom docker image, but even then the runtime itself continues to use the version of python 3 which is installed as part of the OS. The easiest way to get to a newer version is to use a newer runtime. If you're sticking...

  • 3 kudos
2 More Replies
Labels