Read data shared using Delta Sharing open sharing (for recipients) FileNotFoundError

WarmCat — Sat, 25 May 2024 12:27:49 GMT

I'm using the docs here: https://docs.databricks.com/en/data-sharing/read-data-open.html#store-creds

However I am unable to read the stored file which is sucessfully created with the following code:

%scala dbutils.fs.put("dbfs:/FileStore/extraction/config.share",""" {"shareCredentialsVersion":1,"bearerToken":"XXXXXXXXXXXXXXXXXX","endpoint":"https://ukwest.azuredatabricks.net/api/2.0/delta-sharing/metastores/XXXXXXXXXXXXXXXXXX","expirationTime":"9999-12-31T23:59:59.999Z"} """)

When I attempt to read back the file with the following example code:

import delta_sharing client = delta_sharing.SharingClient(f"dbfs:/FileStore/extraction/config.share") client.list_all_tables()

The following error is received:

FileNotFoundError: [Errno 2] No such file or directory: '/databricks/driver/dbfs:/FileStore/extraction/config.share' File <command-3352291941134744>, line 2 1 import delta_sharing ----> 2 client = delta_sharing.SharingClient(f"dbfs:/FileStore/extraction/config.share") 3 client.list_all_tables() File /local_disk0/.ephemeral_nfs/envs/pythonEnv-fe1622b9-cf42-4ef3-9ffc-a7ec8a5b3a2e/lib/python3.11/site-packages/fsspec/implementations/local.py:360, in LocalFileOpener._open(self) 358 if self.f is None or self.f.closed: 359 if self.autocommit or "w" not in self.mode: --> 360 self.f = open(self.path, mode=self.mode) 361 if self.compression: 362 compress = compr[self.compression]

The file browser in Catalog, DBFS shows that the file is present.
What is going on here? Also why is the documentation incorrect?

Thanks in advance

Wayne

Re: Read data shared using Delta Sharing open sharing (for recipients) FileNotFoundError

daniel_sahal — Mon, 27 May 2024 07:33:38 GMT

@WarmCat
As it states in the documentation, try reading the file from

/dbfs/<dbfs-path>/config.share

not from

dbfs:/<dbfs-path>/config.share

Re: Read data shared using Delta Sharing open sharing (for recipients) FileNotFoundError

WarmCat — Mon, 27 May 2024 10:01:54 GMT

Thanks Daniel, but that does not work either.
The only thing that allowed progress was to use:

client = delta_sharing.SharingClient(f"file:///tmp/config.share")

I gave up and installed Apache Spark locally in a venv for now, and will be using AWS going forward.

Not a great experience with Databricks, for example even trying to preview this post results in an error!

Anyway, thanks for your efforts,

Wayne.

topic Re: Read data shared using Delta Sharing open sharing (for recipients) FileNotFoundError in Data Engineering

Read data shared using Delta Sharing open sharing (for recipients) FileNotFoundError

Re: Read data shared using Delta Sharing open sharing (for recipients) FileNotFoundError

Re: Read data shared using Delta Sharing open sharing (for recipients) FileNotFoundError