topic Re: Connecting from Databricks to Network Path in Data Engineering

Connecting from Databricks to Network Path

MathewDRitch — Tue, 16 Apr 2024 11:29:07 GMT

Hi All,

Will appreciate if someone can help me with some references links on connecting from Databricks to external network path. I have Databricks on AWS and previously used to connect to files on external network path using Mount method. Now Databricks does not recommend mounts but recommends Unity Catalog Volumes, but there are no references of using Unity Catalog Volumes to connecting on-premise network folders. Can someone guide in case that it can be achieved?

Re: Connecting from Databricks to Network Path

-werners- — Tue, 16 Apr 2024 15:14:56 GMT

how did you connect to the storage? did you mount it on the cluster with an init script?

Re: Connecting from Databricks to Network Path

MathewDRitch — Tue, 16 Apr 2024 15:50:41 GMT

Currently we connect to the cloud storages as external storages using unity catalog. We have not yet connected to the on-premise network storages, which is currently the solution we are looking for.

Re: Connecting from Databricks to Network Path

-werners- — Wed, 17 Apr 2024 07:15:10 GMT

I don't think that it is possible at the moment. UC focuses on cloud data.
You might want to try to use Minio, but apparently UC does not support Minio yet
Pity, because that would be an awesome solution.

Re: Connecting from Databricks to Network Path

faisaljaved1 — Fri, 27 Dec 2024 15:37:09 GMT

Were you able to access external network path ?

I have a similar requirement where i have to write data from databricks to a network location where other external systems also access and update that file, I am new to databricks and have spend considerable time with no success.

Re: Connecting from Databricks to Network Path

om_khade — Wed, 26 Feb 2025 08:36:31 GMT

Do we have any update on this?