I was able to resolve the above dependency issues by explicitly depending on hadoop-aws 3.4.1 in addition to `spark-hadoop-cloud` and was able to read and write to tables in UC from OSS spark!
However, I'm now having difficulty with the Uniform part. If I only add a dependency on `iceberg-spark-runtime-3.5`, then writing to a Uniform iceberg table fails with
ERROR org.apache.spark.sql.delta.DeltaLog - Failed to find Iceberg converter class
java.lang.ClassNotFoundException: org.apache.spark.sql.delta.icebergShaded.IcebergConverter
If I add `delta-iceberg` then the IcebergConverter class is found, reading and writing to the table works, however the iceberg conversion on write fails with
ERROR org.apache.spark.sql.delta.icebergShaded.IcebergConverter - Error when converting to Iceberg metadata
org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: [SCHEMA_NOT_FOUND] The schema `test_schema` cannot be found. Verify the spelling and correctness of the schema and catalog.
[info] If you did not qualify the name with a catalog, verify the current_schema() output, or qualify the name with the correct catalog.
Possibly related, while `SHOW SCHEMAS` does list the `test_schema` I've been using and `SELECT current_schema()` does indeed return `test_schema`, `spark.sessionState.catalog.listDatabases()` does not! Reading the code, it seems like this is what's referenced during the iceberg conversion. It's not clear to me why the schema / db is not consistent across various spark calls.