Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-18-2024 11:44 PM
hard to tell, do you use unity catalog? have you defined storage credentials/external storage etc?
The reason show() and display() give the error is because of the lazy evaluation of spark. So the error could be somewhere in the code before those actions.
But it could also be a configuration issue.
If you use UC, you first have to make sure it is set up correctly.
If not: make sure you have mounted your storage, or at least test if you can list files (dbutils.fs.ls).
Make sure databricks has permissions to read/write to the storage account.