brickster_2018
Databricks Employee
Databricks Employee

This is working as expected. For Delta streaming, the data files created in the first place will be used for streaming. The optimized files are not considered the downstream streaming job.

This is the reason it's not recommended to run VACUUM with fewer retention hours

View solution in original post