Queries from Serverless compute referring to older/deleted/vacuumed version of the delta tables.

[s_agarwal]

Hi Team,

I have a unity catalog based managed delta table which I am able to successfully query using the regular compute/cluster options.

But when I try to query the same table using a Serverless/SQL Warehouse, they are referring to an older version / metadata of the table which does not exists and I get an error stating that the path does not exists.

In my databricks UI, in the details section of the same table, I see that the property "lastUpdateVersion" points to the exact version which Serverless is trying to read. But my other (non-serverless) computes are reading correct data.

Any help/insight would be appreciated.

[Saritha_S]

Hi @s_agarwal 

Please find below my findinsg for your query. 

• Serverless uses cached Unity Catalog metadata

• Your UC metadata points to an old Delta version

• Regular clusters bypass this cache

• Fix by refreshing or forcing UC metadata rewrite