using the SYNC TABLE command with AWS Glue tables

[JameDavi_51481]

According to the docs, we should be able to use SYNC TABLE to 'upgrade' a table from a Hive metastore to Unity Catalog. We are using AWS Glue as our Hive Metastore, but tables created in Glue do not seem to be set up in a format that Databricks likes. When we try to run `SYNC TABLE` on a Glue table, DBSQL returns the error message `[UPGRADE_NOT_SUPPORTED.HIVE_SERDE] Table is not eligible for upgrade from Hive Metastore to Unity Catalog. Reason: Hive SerDe table.`

Has anyone successfully gotten SYNC TABLE or SYNC SCHEMA working where a Glue table is the source?

[Kaniz_Fatma]

Hi @JameDavi_51481, Unity Catalog does not support tables created in AWS Glue using Hive SerDe and cannot be upgraded using the SYNC command. The recommended solution is to change the tables into Delta format and then issue the SYNC command to upgrade. Alternatively, you can create a new table in Unity Catalog using the CREATE TABLE command and then copy the data from the Glue table using INSERT INTO base. However, this may not preserve all the metadata and settings of the original table.

Sources:
- https://docs.databricks.com/data-governance/unity-catalog/migrate.html#upgrading-tables-from-hive-me...
- https://docs.databricks.com/data/data-sources/aws/amazon-glue-metastore.html#limitations