Is it possible to use Liquid Clustering on Delta Live Tables / Materialised Views?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-04-2024 05:40 AM
Is it possible to use Liquid Clustering on Delta Live Tables?
If it is available what is the Python syntax for adding liquid clustering to a Delta Live Table / Materialised view please?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-04-2024 10:49 AM
Delta Live Tables are not actully tables, it is a pipeline tool that manage delta tables. you can enable liquid clustering on the delta tables that are used by DLT. am not sure about MV, but I think it should work since MVs are materialised as detla table too. but maybe you can try it and let us know the result.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-20-2024 12:23 PM - edited 11-20-2024 12:24 PM
Hi @amr, materialised views are not tables, they are views. Liquid clustering is not supported on views so it will throw [EXPECT_TABLE_NOT_VIEW.NO_ALTERNATIVE] error.
Unfortunately it will be the same case for the "optimize" command as well.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-21-2025 03:51 AM
what is the Python syntax for adding liquid clustering to a Delta Live Table???
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-21-2025 04:46 PM
@Dlt.table(
name=table_name,
comment="just_testing",
table_properties={
"quality": "gold",
"mergeSchema": "true"
},
cluster_by=["test_id", "find_date"] # Optimizes for queries filtering on these columns
)
def testing_table():
return create_testing_table(df_final, test_type)