cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Liquid Clustering limitation clustering on write does not support source queries that include filter

ShivangiB
New Contributor II

I have a query :

%sql
insert into ucdata.brz.liquidcluster_table_data
select sum(col1) as col1,col2,sum(col3) as col3 from ucdata.brz.liquidcluster_table_data  group by col2
This query I am running with run time version 13.3 and it is still working.
 
But as per databricks limitation doc it should not work.
Can you please help.
3 REPLIES 3

SantoshJoshi
New Contributor II

Hi @ShivangiB,

Can you share the definition of your table - ucdata.brz.liquidcluster_table_data?

Curious - why your source and target are the same table? Are you trying to store the different grains of data in the same table?

I think you are referring this article - Use liquid clustering for Delta tables - Azure Databricks | Microsoft Learn

SantoshJoshi_0-1741478498885.png

ShivangiB
New Contributor II

Hey @SantoshJoshi , I am just trying too test the limitations specified for some case study. And while testing can't see these limitations so wanted to understand.

it is having 3 columns , col1,col2,col3 having random numbers and col2 is cluster by column.

ShivangiB
New Contributor II

Hey Team, can you please help on this

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group