cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Read data shared using Delta Sharing open sharing (for recipients) FileNotFoundError

WarmCat
New Contributor II

I'm using the docs here: https://docs.databricks.com/en/data-sharing/read-data-open.html#store-creds

However I am unable to read the stored file which is sucessfully created with the following code:

%scala
dbutils.fs.put("dbfs:/FileStore/extraction/config.share","""
{"shareCredentialsVersion":1,"bearerToken":"XXXXXXXXXXXXXXXXXX","endpoint":"https://ukwest.azuredatabricks.net/api/2.0/delta-sharing/metastores/XXXXXXXXXXXXXXXXXX","expirationTime":"9999-12-31T23:59:59.999Z"}
""")

When I attempt to read back the file with the following example code:

import delta_sharing
client = delta_sharing.SharingClient(f"dbfs:/FileStore/extraction/config.share")
client.list_all_tables()

The following error is received:

FileNotFoundError: [Errno 2] No such file or directory: '/databricks/driver/dbfs:/FileStore/extraction/config.share'
File <command-3352291941134744>, line 2
      1 import delta_sharing
----> 2 client = delta_sharing.SharingClient(f"dbfs:/FileStore/extraction/config.share")
      3 client.list_all_tables()
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-fe1622b9-cf42-4ef3-9ffc-a7ec8a5b3a2e/lib/python3.11/site-packages/fsspec/implementations/local.py:360, in LocalFileOpener._open(self)
    358 if self.f is None or self.f.closed:
    359     if self.autocommit or "w" not in self.mode:
--> 360         self.f = open(self.path, mode=self.mode)
    361         if self.compression:
    362             compress = compr[self.compression]

The file browser in Catalog, DBFS shows that the file is present.
What is going on here? Also why is the documentation incorrect?

Thanks in advance

Wayne

2 REPLIES 2

daniel_sahal
Esteemed Contributor

@WarmCat 
As it states in the documentation, try reading the file from 

/dbfs/<dbfs-path>/config.share

not from

dbfs:/<dbfs-path>/config.share

 

WarmCat
New Contributor II

Thanks Daniel, but that does not work either.
The only thing that allowed progress was to use:

client = delta_sharing.SharingClient(f"file:///tmp/config.share")

I gave up and installed Apache Spark locally in a venv for now, and will be using AWS going forward.

Not a great experience with Databricks, for example even trying to preview this post results in an error!

Anyway, thanks for your efforts, 

Wayne.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!