break production using a shallow clone

[oosterhuisf]

Hi,

If you create a shallow clone using the latest LTS, and drop the clone using a SQL warehouse (either current or preview), the source table is broken beyond repair. Data reads and writes still work, but vacuum will remain forever broken. I've attached a notebook that demonstrates this behaviour.

To me it looks like the 'drop table' command in the warehouse does not remove the reference to the source files.

Wanted solution:
* fixed `drop table` in the SQL Warehouse
* working delta table after `repair table`

[Kaniz]

Hi @oosterhuisf, When you create a shallow clone using the latest LTS and subsequently drop the clone using a SQL warehouse (either current or preview), the source table becomes irreparably broken. Although data reads and writes continue to function, the vacuum operation remains permanently impaired. It appears that the ‘drop table’ command in the warehouse fails to remove the reference to the source files.

Your desired solution involves:

1. Addressing the issue with the drop table command in the SQL Warehouse.
2. Ensuring that the delta table remains functional after executing the repair table.

For more detailed information on shallow and deep clones, you can refer to the official documentation for Azure Databricks SQL. Shallow clones copy only the metadata of the source table, while deep clones create an independent c.... Keep in mind that shallow clones are useful for scenarios like data migration, archiving, machine le....

If you encounter any further issues or need additional assistance, feel free to ask! 😊

[oosterhuisf]

To add to that: the manual does not state that this might happen