I've been struggling with figuring out how to delete a managed delta live table. If I run a drop command in Databricks SQL I get:
[STREAMING_TABLE_OPERATION_NOT_ALLOWED.DROP_DELTA_LIVE_TABLE] The operation DROP is not allowed: The operation does not apply to Streaming Tables created from Delta Live Tables, instead remove the Streaming Table from the pipeline definition in Delta Live Tables.
If I go to the Data section and click the 3 dots and Delete, it only deletes it from the UI. The underlying data is all there. A subsequent run will show all the old data again.
The only solution I've been able to do is to create a new DLT pipeline, and a new target schema, and then re-point my DLT process to them. And then run. There must be a better way. What do I do to be able to completely remove the underlying data of a DLT table?
