cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Read data shared using Delta Sharing open sharing (for recipients) FileNotFoundError

WarmCat
New Contributor II

I'm using the docs here: https://docs.databricks.com/en/data-sharing/read-data-open.html#store-creds

However I am unable to read the stored file which is sucessfully created with the following code:

%scala
dbutils.fs.put("dbfs:/FileStore/extraction/config.share","""
{"shareCredentialsVersion":1,"bearerToken":"XXXXXXXXXXXXXXXXXX","endpoint":"https://ukwest.azuredatabricks.net/api/2.0/delta-sharing/metastores/XXXXXXXXXXXXXXXXXX","expirationTime":"9999-12-31T23:59:59.999Z"}
""")

When I attempt to read back the file with the following example code:

import delta_sharing
client = delta_sharing.SharingClient(f"dbfs:/FileStore/extraction/config.share")
client.list_all_tables()

The following error is received:

FileNotFoundError: [Errno 2] No such file or directory: '/databricks/driver/dbfs:/FileStore/extraction/config.share'
File <command-3352291941134744>, line 2
      1 import delta_sharing
----> 2 client = delta_sharing.SharingClient(f"dbfs:/FileStore/extraction/config.share")
      3 client.list_all_tables()
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-fe1622b9-cf42-4ef3-9ffc-a7ec8a5b3a2e/lib/python3.11/site-packages/fsspec/implementations/local.py:360, in LocalFileOpener._open(self)
    358 if self.f is None or self.f.closed:
    359     if self.autocommit or "w" not in self.mode:
--> 360         self.f = open(self.path, mode=self.mode)
    361         if self.compression:
    362             compress = compr[self.compression]

The file browser in Catalog, DBFS shows that the file is present.
What is going on here? Also why is the documentation incorrect?

Thanks in advance

Wayne

2 REPLIES 2

daniel_sahal
Esteemed Contributor

@WarmCat 
As it states in the documentation, try reading the file from 

/dbfs/<dbfs-path>/config.share

not from

dbfs:/<dbfs-path>/config.share

 

WarmCat
New Contributor II

Thanks Daniel, but that does not work either.
The only thing that allowed progress was to use:

client = delta_sharing.SharingClient(f"file:///tmp/config.share")

I gave up and installed Apache Spark locally in a venv for now, and will be using AWS going forward.

Not a great experience with Databricks, for example even trying to preview this post results in an error!

Anyway, thanks for your efforts, 

Wayne.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group