cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

using the SYNC TABLE command with AWS Glue tables

JameDavi_51481
Contributor

According to the docs, we should be able to use SYNC TABLE to 'upgrade' a table from a Hive metastore to Unity Catalog. We are using AWS Glue as our Hive Metastore, but tables created in Glue do not seem to be set up in a format that Databricks likes. When we try to run `SYNC TABLE` on a Glue table, DBSQL returns the error message `[UPGRADE_NOT_SUPPORTED.HIVE_SERDE] Table is not eligible for upgrade from Hive Metastore to Unity Catalog. Reason: Hive SerDe table.`

Has anyone successfully gotten SYNC TABLE or SYNC SCHEMA working where a Glue table is the source?

1 REPLY 1

Vidhi_Khaitan
Databricks Employee
Databricks Employee

Hi team,
The issue arises because Unity Catalog's SYNC command cannot process tables that were created using the Hive SerDe format in the hive_metastore catalog. This means that any attempts to upgrade such tables will fail.

You need to convert your Hive SerDe tables to Delta format using the following command:
CONVERT TO DELTA hive_metastore.<database-name>.<table-name>;

Then after converting the table to Delta format, you can issue the SYNC command to upgrade the table to Unity Catalog using:
SYNC hive_metastore.<database-name>.<table-name> TO unity_catalog.<database-name>.<table-name>;

Let me know if this helps!

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local communityโ€”sign up today to get started!

Sign Up Now