โ02-03-2022 08:43 AM
Hi, Iโm trying to read file from S3 root bucket. I can ls all the files but I canโt read it because of access denied. When I mount the same S3 root bucket under some other mountPoint, I can touch and read all the files. I also see that this new mountPoint have an empty encryptionType while others (including root) have sse-s3.
Maybe this could be the issue (we are trying access data through instance profiles)? My goal is to make root mountPoint "/" accessible because of all our hardcoded paths in dependencies and modules. We already checked all IAM roles, instance_profiles, bucket policies but without any luck.
Additional info:
Is there something we are missing? Thank you for your help!
โ03-16-2022 02:01 AM
โ02-03-2022 09:26 AM
Hi @Duลกan Vystrcilโ. My name is Piper, and I'm one of the moderators here in the community. Thanks for your question and it's great to meet you. ๐
We will give the community a chance to respond and then we will come back around if we need to.
โ02-12-2022 08:33 AM
As databricks has the ownership of rootbucket I think manually upload from console will not work.For your second point - ( import .zip file as .whl library and %sh unzip ) I hope you are doing this through notebook so it will not giving you any issue.
โ02-16-2022 08:56 AM
@Duลกan Vystrcilโ - Checking in. Does Atanu's answer help you?
โ02-17-2022 12:05 AM
Hi @Atanu Sarkarโ , @Piper Wilsonโ ,
โ
thanks for the replies. Well I don't understand the fact about ownership. I believe that rootbucket is still under my ownership (I created it and I could upload/delete any files through browser without any problem). I can even list those files from notebook but can't read them.
โ
I discovered that if I upload those files through databricks cli, I could read them. So maybe you're right about the "ownership" but I can't find any docs regarding this, so I could understand it. Do we have anything?
โ03-16-2022 02:01 AM
โ03-22-2022 09:41 AM
Hi @Duลกan Vystrcilโ ,
Just a friendly follow-up. Did you have time to check the docs @Kaniz Fatmaโ shared? did it help to resolve your issue or not? please let us know.
โ03-22-2022 10:18 AM
Yeah Jose, that was exactly I was looking for! Thanks @Kaniz Fatmaโ
โ03-23-2022 07:49 AM
You're welcome @Duลกan Vystrcilโ . Happy to help you. Would you like to mark it as the best answer which would turn out to helpful for the entire community?
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.