Can anybody confirm if external volumes pointing to S3 access points work in Databricks on AWS?
- I have S3 bucket, but can only access it via S3 access point. The bucket is KMS encrypted.
- I created an IAM role that can list and read the S3 access point (and can also use the KMS key, plus it gives read access to the underlying bucket). I double checked that it can browse the S3 access point.
- The IAM role is assumable by Databricks and by itself.
- I registered a storage credential and defined an external location (using the former)
- I created an external volume that uses the very same external location, and I have READ VOLUME privilege
With that:
- I can browse the files (of the S3 access point) using the external location; however
- When I try to browse files via the external volume, I get "Access to the storage bucket is forbidden by AWS." error.
I would assume that if I can browse the S3 access point via the external location, I would also be able to browse it via the (linked) external volume. What am I doing wrong? Do S3 access points work for external volumes?
