cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

removal of Delta live tables

Anish_2
New Contributor II

Hello Team,

I have removed definition of table from delta live table pipeline but table is still present in unity catalog. In event log, it is giving below message

Materialized View '`catalog1`.`schema1`.`table1`' is no longer defined in the pipeline and has been marked as inactive in Unity Catalog.
This table will no longer be updated by the pipeline on the next update.
If you want this table to be updated by the pipeline on the next update, you can add the table definition back to the pipeline
which will make the table active again.

Could you please help me why table is marked inactive instead of removal?

 

3 REPLIES 3

KaranamS
Contributor II

In DLT, when you remove table definition, it is not automatically dropped from Unity Catalog. This is to prevent any accidental data loss and reactivate easily if needed. If you want the table to be deleted from UC, manual cleanup is required.

Brahmareddy
Honored Contributor II

Hi @Anish_2 

How are you doing today? I agree with @KaranamS's answer.

Databricks marks the table as inactive instead of removing it to prevent accidental data loss, allowing you to restore it if needed. Once inactive, the table remains in Unity Catalog but won’t be updated by the pipeline. To fully remove it, you need to manually run DROP TABLE catalog1.schema1.table1; in SQL. If it’s a managed table, use DROP TABLE ... PURGE; to also delete storage files. Databricks doesn’t auto-remove tables when they are removed from the pipeline, so cleanup must be done manually.

Give a try and let me know if this works!

Good day.

Regards,

Brahma

Anish_2
New Contributor II

We are using on job cluster and i have observed that configuration pipelines.dropInactiveTables false  is coming under advanced configuration of cluster. This is coming after latest release of databricks. While creating DLT, i added this property as True then databricks is removing table instead of marking it as inactive. Can you guide me if we can use same property in cluster policy definition?