Create external table using iceberg not working

[new_user_ptl]

I'm trying to create a external iceberg table on top of files I have in s3 created using a spark job that runs outside of databricks using Hadoop catalog.

Create External table table_name using iceberg location 's3://myicebergbucket/iceberg_table_path'

But I'm getting this error

Managed iceberg tables do not support replace on external tables.

Is there a solution to this?

[juan_maedo]

Hey! So you're thinking about creating an Iceberg table as an external table on Unity Catalog? Yeah, there are definitely some limitations to keep in mind—especially around writes and creation, since it's still in preview.

he main thing is: you can read Iceberg tables from other engines through UC external tables, but you can't write to them that way. Write access is only available for managed tables right now. Check out the https://docs.databricks.com/aws/en/external-access/iceberg#requirements

So you can read Iceberg table from other engines, but can't write vía UC using external tables, only available for managed tables.

That said, if you're already creating Iceberg tables in another engine, you can still access them through UC using LHF (Lakehouse Federation). That way you can read them without hitting those external table write limitations. https://docs.databricks.com/aws/en/iceberg/#read-iceberg-tables-managed-by-other-catalogs

Hope that helps!

[Raman_Unifeye]

External Write yet to be supported for iceberg

[new_user_ptl]

It's ok if I'm not able to write to the table. The problem is I'm not even able to create the external table in Databricks. 

I was trying to create on top of the s3 location. That s3 location contains both data and the Metadata needed for iceberg. And I was not using any foreign catalog. It's a spark job that writes data to s3 using iceberg format.

[saurabh18cs]

Hi @new_user_ptl Databricks does NOT support directly registering or querying Iceberg tables that use the Hadoop catalog (where metadata is stored only as files in S3 and not registered in any metastore/catalog). when use databricks sql , registering table require the table to be discoverable via catalog glue or hive etc but with hadoop catalog it is just s3 files.

You can still try readng the data using Spark directly (not via SQL), for example

spark.read.format("iceberg").load("s3://bucket/path")