Is there a way to register S3 compatible tables?

[tabasco]

Hi everyone,
I have successfully registered AWS S3 tables in unity catalog, but I would like to register S3-compatible as well.

But, to create an EXTERNAL LOCATION in unity catalog, it seems I must register a credential. But the only suported credential type for S3 is IAM Role ARN. Since S3-compatible systems don't have a role ARN, it appears unity catalog does not support S3-compatible external locations.

Is there another way to register S3-compatible tables in UC?

Thank you in advance for any guidance.

[BS_THE_ANALYST]

Hey @tabasco, did you check out the External Table documentation in the Databricks AWS docs?

External Location
https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-external-locations

Credentials
https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-storage-credentials

@tabasco it does appear that it's supporting an AWS IAM role. What would you be looking for instead?

All the best,
BS

[tabasco]

Thank you for your kind response.

My point is that for S3-compatible storage, we can access objects with an S3 endpoint, access key, and secret key instead of using a roleARN.

But currently, it seems that only roleARN are supported for creating credentials.

Also, I did see a similar question before, but that was from two years ago, so I was just wondering if there have been any updates since there.

https://community.databricks.com/t5/administration-architecture/setup-unity-catalog-external-locatio...

[szymon_dybczak · Accepted solution]

Hi @tabasco ,

And I can confirm. If you want to use S3 storage in UC the storage needs to support roleARN. I think you can still try to use S3-compatible storage in the old way - via mounts.

[tabasco]

Hi, @szymon_dybczak 

Thank you for your kind reply.
It was very helpful for me.

Have a Good day !