How to access root mountPoint without "Access Denied"?

Dusko
Databricks Partner

Hi, I’m trying to read file from S3 root bucket. I can ls all the files but I can’t read it because of access denied. When I mount the same S3 root bucket under some other mountPoint, I can touch and read all the files. I also see that this new mountPoint have an empty encryptionType while others (including root) have sse-s3.

Maybe this could be the issue (we are trying access data through instance profiles)? My goal is to make root mountPoint "/" accessible because of all our hardcoded paths in dependencies and modules. We already checked all IAM roles, instance_profiles, bucket policies but without any luck.

Additional info:

  • Files I want to read are uploaded manually through AWS console
  • When I import .zip file as .whl library and %sh unzip that file I can read this file without any problem

Is there something we are missing? Thank you for your help!