Databricks repo not working with installed python libraries

NelsonE — Mon, 26 Aug 2024 20:56:53 GMT

Hello,

I'm trying to use some installed libraries in my cluster.

I created a single node cluster with the version Runtime version 14.3 LTS.

I also installed libraries like oracledb==2.2.1

Then when I try to use python to load this libraries in the workspace/shared folder works perfectly, when I try to run it on my repo (github folder) I'm not able to load any libraries. I'm getting this error:

OSError: [Errno 5] Input/output error: '/Workspace/Users/nelson.candia@farmatodo.com/ftd-databricks-repo/libraries' File <command-3142143133324967>, line 1 ----> 1 import oracledb 2 import pandas as pd File <frozen importlib._bootstrap_external>:1659, in _fill_cache(self)

Also I'm able to see in the left side panel that the library is installed, even if I do a pip list I can see the library is installed.

Re: Databricks repo not working with installed python libraries

Brahmareddy — Thu, 29 Aug 2024 03:00:52 GMT

Hello Nelson, How are you dong today?

Try checking the permissions on your repo folder to ensure your cluster can access it without issues. Use absolute paths when running from your GitHub repo to avoid directory confusion. Reinstall the oracledb library directly in your repo environment to make sure it's accessible. Restart the cluster after installing libraries to ensure everything loads correctly. Finally, check for any path conflicts in your repo environment that might be causing problems.

Give a try and let me know if it works.

Regards,

Brahma

topic Re: Databricks repo not working with installed python libraries in Get Started Discussions

Databricks repo not working with installed python libraries

Re: Databricks repo not working with installed python libraries