cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

Refresh a External table metadata

Dp15
Contributor

Hi,

I have an external table which is created out of a S3 bucket. The first time I am creating the table I am using the following command :

 

query = """CREATE TABLE IF NOT EXISTS catalog.schema.external_table_s3
           USING PARQUET
           LOCATION '{}'""".format(bucket_location)
 
The first time the table is created the files in the 'bucket_location' are loaded into the table. Now, I have a use case where when new files are added in the S3, I would like the metadata of this external table to be refreshed. Is it possible to refresh the metadata of the external table? I tried using the MSCK Repair command 

MSCK REPAIR TABLE catalog.schema.external_table_s3

but my command failed with the following error:
 
AnalysisException: [UC_COMMAND_NOT_SUPPORTED.WITHOUT_RECOMMENDATION] The command(s): Repair Table are not supported in Unity Catalog. ;
RepairTableCommand catalog.schema.external_table_s3, true, false, MSCK REPAIR TABLE
 
1 ACCEPTED SOLUTION

Accepted Solutions

Kaniz
Community Manager
Community Manager
3 REPLIES 3

Kaniz
Community Manager
Community Manager

Dp15
Contributor

Hi @Kaniz thank you for reply, how can we handle the schema changes in the external location, what if there are additions or deletions on the schema, will the refresh table work then too? 

Kaniz
Community Manager
Community Manager
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!