- 29197 Views
- 10 replies
- 2 kudos
File not found error.
FileNotFoundError: [Errno 2] No such file or directory: '/dbfs/FileStore/config.share'When I am trying to read the config.share file, it is throwing this error. I tried with spark path format as well which is dbfs:/FileStore/config.share' but it also...
- 29197 Views
- 10 replies
- 2 kudos
- 2 kudos
On Unity Catalog Shared Access Mode clusters you need to use a UC Volume to read (config) files using vanilla Python (with open() for example that many libs use). You can no longer read files from DBFS this way. This is all part of the new security m...
- 2 kudos
- 6000 Views
- 4 replies
- 15 kudos
How would i upload file stream object to S3 bucket using pyspark?
I could able to save data using pyspark into S3 but not sure on how to save a file stream object into S3 bucket using pyspark. I could achieve this with help of python but when Unity catalog was enabled on Databrciks it always ends up with an access ...
- 6000 Views
- 4 replies
- 15 kudos
- 15 kudos
I got to know that there is a change required at Unity-catalog to make it work with Python and got a recommendation to use pyspark to store file into S3.I do not see much information about storing a file stream object in an S3 bucket anywhere. Can an...
- 15 kudos