Hi @Zachary Higgins​ 

Thanks for the reply

Currently, we are also using Trigger.once so that we can handle the merge stream dependencies properly. But was wondering whether we can scale our pipeline to be streaming by changing the Trigger duration in the future

Thanks for the heads up regarding Concurrency, I was assuming a Pessimistic concurrency control where delta would handle concurrent writes to the same partition without exceptions or data corruption.

I don't think we can support full streaming pipelines if there can be concurrency issues