topic Re: How to configure storage credentials for aws s3 buckets which use access keys in Data Governance

How to configure storage credentials for aws s3 buckets which use access keys

karthik_p — Thu, 04 Jan 2024 02:31:16 GMT

We have a requirement to configure storage credentials in UC for s3 buckets which have been previously mounted by using access keys

mounts has been depreciated and we need to configure storage credentials when we use UC. But while configuring storage creds it only supports instance profiles

Re: How to configure storage credentials for aws s3 buckets which use access keys

karthik_p — Thu, 04 Jan 2024 21:20:35 GMT

@Retired_mod was looking for below scenario https://docs.gcp.databricks.com/en/connect/storage/amazon-s3.html

here buckets are configured with Access keys as mounts, now workspace configured with UC and UC won't recommend to use DBFS mounts, we need to convert them as storages. where as in storage we have only IAM role option (Instance profile) , but there is no option to specify aws access keys to create storage credentials. within article no where it was mentioned as Depreciated.

we are looking for article where it specifies above scenarios are not supported, some thing like that. This is blocking us to proceed with UC upgrade. please help to resolve this

Re: How to configure storage credentials for aws s3 buckets which use access keys

nkraj — Tue, 16 Jan 2024 18:20:08 GMT

Hello @karthik_p ,

The UC setup provides the feasibility to add the S3 bucket as a volume when compared to the non-UC concept Mounts.

When added as Volume, the access to the Volume in UC is Governed via the Storage Credentials.

Regarding Storage credential, as by definition : A storage credential is a securable object representing an AWS IAM role, which itself suggests that it needs to be an IAM role, hence ruling out other authentication mechanisms.

Additionally as mentioned in the article: "A storage credential represents an authentication and authorization mechanism for accessing data stored on your cloud tenant, using an IAM role." It further establishes that the UC Storage credential is expected to be defined using an IAM role.