Hi @Uj337,

How are you doing today?

This issue seems to be tied to the private network setup affecting access to the .whl file on DBFS. i recommend you to start by ensuring the driver node has proper access to the dbfs:/Volumes/any.whl path and that all permissions are correctly configured. If this doesn’t resolve it, consider hosting the .whl file in an accessible cloud storage location, such as Azure Blob or S3, and update your job to reference it there. Additionally, i recommend you to check your network and cluster settings to ensure they allow access to the storage location and that the driver node isn't resource-constrained. Since this is an intermittent issue, you can implement a retry mechanism for your job to handle these temporary failures. If the problem persists, reaching out to Databricks support with the error code and cluster details might help identify a solution.

Give a try and let me know for any queries.

Thanks,

Brahma