I was trying to install a personalized .whl file located in the "shared" folder but I'm obtaining this error:
org.apache.spark.SparkException: Process List(/bin/su, libraries, -c, bash /local_disk0/.ephemeral_nfs/cluster_libraries/python/python_start_clusterwide.sh /local_disk0/.ephemeral_nfs/cluster_libraries/python/bin/pip install --upgrade /local_disk0/tmp/addedFilef07dccf16acf40d19c1a6efc5fd872ec7514431202893444442/dbx_my_process-0.2.6-py3-none-any.whl --disable-pip-version-check) exited with code 1. ERROR: Could not install packages due to an OSError: [Errno 1] Operation not permitted: '/Workspace/Shared/Libraries/my-library-0.2.7-py3-none-any.whl'
The pyproject.toml installs the wheel this way (I'm using dbx and uploading it to a workflow):
[tool.poetry.dependencies]
python = ">=3.9,<3.11"
my-library = {file = "/Workspace/Shared/Libraries/my-library-0.2.7-py3-none-any.whl"}
