I am using DLT and unity catalog and using managed tables. The first table in this pipeline is a live streaming table. I first did this in the SQL editor:
DELETE FROM my_table;
This appears to have deleted all the records, which I wanted since now when I do a SELECT, it shows 0 rows. Then I re-ran the pipeline. This first table is running this command:
CREATE OR REFRESH STREAMING LIVE TABLE my_table
COMMENT "Data from S3"
TBLPROPERTIES ("myCompanyPipeline.quality" = "bronze")
AS
SELECT * FROM cloud_files("s3://my-bucket/path/", "parquet");
However when this pipeline finishes, the table has no data in it. What am I not understanding?
