Limitation as of now in delta live table

[User16826994223]

I am thinking of using delta live table, before that I want to be aware of the limitations it has as of now when it s announced on datasummit 2021

[Anonymous · Accepted solution]

It currently supports Python and SQL. The following functionalities are currently not supported with Delta Live Tables: 

• updates/deletes (MERGE)
• field level lineage
• auto-scaling

[PeteC]

Also found out today that you can't delete any data from your delta live tables or else the process fails with an error:

......

This is currently not supported. If you'd like to ignore deletes, set the option 'ignoreDeletes' to 'true'.

It's not possible to set the ignoreDeletes to true either.

So, unless you're happy to keep the data in your Delta Live Tables forever, you've got a little problem. We're looking at workarounds but nothing pretty yet.

[Zachary_Higgins]

There doesn't appear to be a way to enforce a retention policy on source tables when defining a structured stream. Setting the options for "ignoreChanges" and "ignoreDeletes" doesn't seem to have any effect at all. CDC does not fill this role either, as it only lets you change sink operations based on source data.

The fact that this STILL doesn't support any sort of data retention/pruning strategy in structure stream sources has us investing a lot of time in figuring how to undo all of our investments into DLT and abandon it all together for regular old structure streams.