Delta Live Tables are dropped when pipeline is deleted

[BobEng]

I created simplistic DLT pipeline that create one table. When I delete the pipeline the tables is dropped as well. That's not really desired behavior. Since I remember there was a strong distinction between data (stored in tables) and processing (spark). It's kind of unexpected that when I delete/recreate my job definition all the associated data is gone as well. I'd expect that after recreating the pipeline it will pick up where it left - existing tables will be imported rather that recreated from scratch.

Has anyone faced the similar issue?

Are there any workaround to make sure that tables are keep intact even when pipeline is deleted?

[Kaniz_Fatma]

Hi @BobEng, Here are a few things that might help:

Pipeline Settings: Delta Live Tables provides a user interface for configuring and editing pipeline .... You can configure most settings with either the UI or a JSON specification. 

Table Management: It’s noted that the DROP TABLE command doesn’t apply to Streaming Tables created f.... Instead, you need to remove the Streaming Table from the pipeline definition in Delta Live Tables. This might be a workaround to ensure your tables remain even when the pipeline is deleted.

Community Discussions: There are discussions in the Databricks community about similar issues. You might find some useful insights or workarounds there.

[Kaniz_Fatma]

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!