Declarative Pipeline Re-Deployment and existing managed tables exception

[crami]

Hi,

I am facing a issue regarding re deployment of declarative pipeline using asset bundle. On first deployment, I am able to run the pipeline successfully. On execution, pipeline, as expected create tables. However, when I try to re-deploy the pipeline, with bug fixes or new transformations, on execution, it throws existing managed table already exist exception. I assume the re deployments should accept the existing tables created earlier and should continue run successfully with additions.

[bianca_unifeye]

• Redeploys with Asset Bundles can safely reuse existing DLT-managed tables.

• If you see “managed table already exists”, it usually means:

  • You’re using plain CREATE TABLE  and you should have OR REFRESH part of it

  • The bundle created a new pipeline pointing at a schema where tables already exist, or

  • You changed the nature of an existing table (table ↔ view, streaming ↔ batch) without dropping/renaming it.

[Poorva21]

Managed tables are “owned” by a DLT pipeline. Re-deploying a pipeline that references the same managed tables will fail unless you either:

1. Drop the existing tables first

2. Use external tables that are not owned by DLT

3. Use a separate development schema/pipeline for testing changes

This is by design to protect data integrity and pipeline ownership.