Hello, we are using Databricks Unity catalog to load an Iceberg table, i.e., Delta Lake table with Uniform feature). We are using this guide: https://docs.databricks.com/en/external-access/iceberg.html. This has been working for us since last year but noticed today that the API for loading a table has a new JSON response format which is no longer compatible with existing Apache Iceberg code. As a result, all our read capabilities are broken. The new JSON format looks like this. Notice the new field "payload".
{
"payload": "{\"metadata-location\":\"........."}"
}
The old JSON format, which is consistent with Apache Iceberg spec for REST catalog looks like this:
{
"metadata-location": "<location>",
"metadata": {<rest of metadata here>}
}
Could you please let me know if the API is versioned or if this change is documented? I've looked around and can't find any. Thanks
