Greetings,
I'm writing this message since I learned that AWS has a storage class that is faster than S3 Standard called "S3 Express One Zone". (https://aws.amazon.com/s3/storage-classes/express-one-zone/)
AWS offers support for this storage class with AWS EMR and similar services (https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-express-one-zone.html)
The connector that in theory should offer the support for this is the hadoop s3a connector (https://issues.apache.org/jira/browse/HADOOP-18996) from version 3.3.7+, however I seen that in the latest databricks runtime 16, the hadoop-client-runtime is on 3.3.6, therefore I assume this is not supported.
In addition I have tried to map the s3 express bucket, also called a "directory bucket", in unity catalog, but unity catalog failed to recognize it as a valid bucket.
Has anyone else experimented with this?
Regards,
Cristian
