CLONE not supported on delta table with Liquid Clu...

seydouHR · ‎12-05-2023

Hello all,

We are building a data warehouse on Unity Catalog and we use the SHALLOW CLONE command to allow folks to spin up their own dev environments by light copying the prod tables.

We also started using Liquid Clustering on our feature tables, though we are running into an error when trying to shallow clone these tables. It seems CLONE command is not allowed for tables that have been liquid clustered.

We'd like to know if this was a temporary restriction / is it in the roadmap at some point in the future to allow a cloning operations on tables that have been liquid clustered ?

Exception :

UnsupportedOperationException: CLONE is not supported for Delta table with Liquid clustering.

Runtime : 13.3

Thanks !

Seydou

seydouHR · ‎12-07-2023

Thanks Kaniz for your reply. I was able to get it make it work using runtime 14.0.

Regards,

View solution in original post

Wolfoflag · ‎03-02-2024

Hi @seydouHR @Retired_mod where you able to shallow clone or deep clone the liquid clustering table in runtime 14.0?

seydouHR · ‎03-03-2024

Hi @Wolfoflag ,

Yes, whithin Databricks I was able to shallow clone using runtime 14.0.

Also, outside Databricks, it worked when using the databricks sdk with a SQL Warehouse of version 2023.50 or above.

Example with the sdk:

WORKSPACE_CLIENT.statement_execution.execute_statement(

warehouse_id=sql_warehouse_id, statement=statement

)

Where sql_warehouse_id is the id of your sql warehouse and

your statement could be something like :

f"CREATE OR REPLACE TABLE {target_table_full_name} SHALLOW CLONE {source_table_full_name}"

WORKSPACE_CLIENT is the workspace client from databricks sdk (

i.e from databricks.sdk import WorkspaceClient

)

CLONE not supported on delta table with Liquid Clustering