How to configure Databricks Connect to 'Assume Role' when accessing file from an AWS S3 bucket?

[lsoewito]

I have a Databricks cluster configured with an instance profile to assume role when accessing an AWS S3 bucket.

Accessing the bucket from the notebook using the cluster works properly (the instance profile can assume role to access the bucket).

However, when accessing the same bucket via Databricks Connect with the same cluster doesn't work. It throws 'InvalidMountException' with message '403 Forbidden'.

[Anonymous]

Hello, @lsoewito​ - My name is Piper, and I'm a moderator for the Databricks community. Welcome and thank you for coming to us with your question. I'm sorry to hear that you're having trouble. Let's give your peers a chance to answer your question. We'll circle back if we need to. 🙂

[Kaniz_Fatma · Accepted solution]

Hi @lsoewito​ , Here's how you can secure access to S3 buckets using instance profiles:-

• https://docs.databricks.com/administration-guide/cloud-configurations/aws/instance-profiles.html

Databricks Connect on High Concurrency clusters is available only in Databricks Runtime 7.3 LTS and above.

[Kaniz_Fatma]

Hi @lsoewito​ , We haven't heard from you since my last response, and I was checking back to see if my suggestions helped you. Or else, If you have any solution, please share it with the community as it can be helpful to others. Also, please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.