How can I use custom python library in Azure Databricks?

Jon
New Contributor II

‎10-20-2021 01:20 AM

I am trying to access functions in my coreapi.py by importing in the main notebook, but I have error

ModuleNotFoundError: No module named 'coreapi'. I tried by uploading the file into the same folder and I tried creating a python egg and uploading it. How can i import my python library?

Labels:
0 Kudos

Prabakar
Databricks Employee
Databricks Employee

‎10-20-2021 01:28 AM

Hi @Jon, you need to install the library on the cluster. Please refer to the below document.

https://docs.databricks.com/libraries/cluster-libraries.html#install-a-library-on-a-cluster

Hubert-Dudek
Databricks MVP

‎10-20-2021 02:35 AM

When you go to cluster to edit or you edit individual task you can add own library:image


My blog: https://databrickster.medium.com/
Preview file
17 KB

-werners-
Esteemed Contributor III

‎10-20-2021 03:01 AM

There is also the possibility to use Repos file functionality:

https://databricks.com/blog/2021/10/07/databricks-repos-is-now-generally-available.html