topic Urgent: Iceberg REST catalog - load a table has new JSON format in Data Engineering

Urgent: Iceberg REST catalog - load a table has new JSON format

ronaldgeneblazo — Tue, 28 Jan 2025 20:45:05 GMT

Hello, we are using Databricks Unity catalog to load an Iceberg table, i.e., Delta Lake table with Uniform feature). We are using this guide: https://docs.databricks.com/en/external-access/iceberg.html. This has been working for us since last year but noticed today that the API for loading a table has a new JSON response format which is no longer compatible with existing Apache Iceberg code. As a result, all our read capabilities are broken. The new JSON format looks like this. Notice the new field "payload".

{
"payload": "{\"metadata-location\":\"........."}"
}

The old JSON format, which is consistent with Apache Iceberg spec for REST catalog looks like this:

{
"metadata-location": "<location>",
"metadata": {<rest of metadata here>}
}

Could you please let me know if the API is versioned or if this change is documented? I've looked around and can't find any. Thanks

Re: Urgent: Iceberg REST catalog - load a table has new JSON format

Satyadeepak — Tue, 28 Jan 2025 21:10:26 GMT

Yes @ronaldgeneblazo - We are aware of this issue. Our Engineering team is looking on to this on priority to get this addressed. Sorry for the inconvenience.

Re: Urgent: Iceberg REST catalog - load a table has new JSON format

ronaldgeneblazo — Thu, 06 Feb 2025 17:17:55 GMT

Satyadeepak - it looks like this has been fixed on your end and we are no longer seeing this issue. Thanks for checking.