cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Use of venv pack

c038644
New Contributor II

Hi, I very new so this probably sounds stupid...

I'm following the blog on How to Manage Python Dependencies in PySpark:

https://www.databricks.com/blog/2020/12/22/how-to-manage-python-dependencies-in-pyspark.html

...but when I try the packing works fine but I recieve a file not found error (see screenshot). What can I do to fix this?

Thanks in advance.

4 REPLIES 4

Hubert-Dudek
Esteemed Contributor III

There are no stupid questions, only ones not asking them 🙂

What package are you need on the cluster? Maybe it can be installed using ready out-of.the-box options.

Debayan
Esteemed Contributor III
Esteemed Contributor III

Can you try using an absolute path instead of a relative path for the same?

For example: https://stackoverflow.com/questions/38661464/filenotfounderror-winerror-3

c038644
New Contributor II

Thanks for your answer, I tried but it didn't make any differerence (see attached). I will look again at my environment settings as as the error is coming back with '\\' maybe I just need to delete a '\' somewhere?

Kaniz
Community Manager
Community Manager

Hi @Peter Scaife​, Thank you for your response. Please keep us posted.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.