cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

How to configure storage credentials for aws s3 buckets which use access keys

karthik_p
Esteemed Contributor

We have a requirement to configure storage credentials in UC for s3 buckets which have been previously mounted by using access keys 

 mounts has been depreciated and we need to configure storage credentials when we use UC. But while configuring storage creds it only supports instance profiles 

2 REPLIES 2

karthik_p
Esteemed Contributor

@Retired_mod was looking for below scenario https://docs.gcp.databricks.com/en/connect/storage/amazon-s3.html

here buckets are configured with Access keys as mounts, now workspace configured with UC and UC won't recommend to use DBFS mounts, we need to convert them as storages. where as in storage we have only IAM role option (Instance profile) , but there is no option to specify aws access keys to create storage credentials. within article no where it was mentioned as Depreciated. 

we are looking for article where it specifies above scenarios are not supported, some thing like that. This is blocking us to proceed with UC upgrade. please help to resolve this  

nkraj
Contributor

Hello @karthik_p , 

The UC setup provides the feasibility to add the S3 bucket as a volume when compared to the non-UC concept Mounts.

When added as Volume, the access to the Volume in UC is Governed via the Storage Credentials.

Regarding Storage credential, as by definition : A storage credential is a securable object representing an AWS IAM role, which itself suggests that it needs to be an IAM role, hence ruling out other authentication mechanisms.

Additionally as mentioned in the article: "A storage credential represents an authentication and authorization mechanism for accessing data stored on your cloud tenant, using an IAM role." It further establishes that the UC Storage credential is expected to be defined using an IAM role.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group