Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
Showing results for 
Search instead for 
Did you mean: 

Error Handling for Web Data Retrieval and Storage in Databricks UNITY Clusters

New Contributor

The following code works well in a normal Databricks cluster, where it passes a null JSON and retrieves content from the web link.

However, in a Unity cluster, it produces the following error: 'FileNotFoundError: [Errno 2] No such file or directory: '/dbfs/mnt/raw/processing/facts/acc/dataacc.json

import requests

data1 ='https://weblink', json={})

with open('/dbfs/mnt/raw/processing/facts/acc/dataacc.json', mode='wb') as dataFile:


why does it create a write error  when using unity cluster?




Honored Contributor
Honored Contributor

Hi @nidhin , Good Day! 

The reason behind the below error while trying to access the external dbfs mount file using "with open" is that you are using a shared access mode cluster.

'FileNotFoundError: [Errno 2] No such file or directory: '/dbfs/mnt/raw/processing/facts/acc/dataacc.json

This is a known limitation for Shared Clusters, where /dbfs path is not accessible. You can try using a single-user cluster instead to access /dbfs which supports UC.

Please refer:

And we also have a preview feature 'Improved Shared Clusters' that addresses some of the limitations of Shared Clusters.

Please let me know if this helps and leave a like if this information is useful, followups are appreciated.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!