- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-16-2025 10:30 PM
Hey folks,
I have a job that runs on a serverless compute. I have also created a wheel file with custom functions, which I require in this job. I see that from here, we cannot install libraries for a task and must use notebook-scoped libraries. So what I am doing right now is:
1. Build the wheel locally
2. Push it to a location in dbfs (dbfs:/datalake/libs)
3. Install the library using %pip install /dbfs/datalake/libs/library.whl
But I get this error:
WARNING: Requirement '/dbfs/datalake/libs/library.whl' looks like a filename, but the file does not exist ERROR: library.whl is not a valid wheel filename.
I if do %fs ls dbfs:/datalake/libs/ i am able to see the wheel file.
But if i do !ls /dbfs/datalake/libs/, i do not see it.
So far, I'm having no luck installing the wheel file from dbfs. Can someone help me with some instructions or suggest a better way?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-18-2025 01:48 PM
Is your dbfs mounted?
Otherwise, try uploading it to your workspace's "shared" folder - this is a common place to put these sorts of files. dbfs is slowly getting phased out and not really in any best practices.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-19-2025 12:32 AM - edited 06-19-2025 12:33 AM
I created a new volume and copied the files in there. works well for me.