https://community.databricks.com/t5/data-engineering/what-is-the-difference-between-a-trigger-once-stream-and-a/m-p/23519#M16244 <P>It seems to me like both of these would accomplish the same thing in the end. Do they use different mechanisms to accomplish it though? Are there any hidden costs to streaming to consider? </P> Thu, 17 Jun 2021 03:34:52 GMT User16826992666 2021-06-17T03:34:52Z https://community.databricks.com/t5/data-engineering/what-is-the-difference-between-a-trigger-once-stream-and-a/m-p/23519#M16244 <P>It seems to me like both of these would accomplish the same thing in the end. Do they use different mechanisms to accomplish it though? Are there any hidden costs to streaming to consider? </P> Thu, 17 Jun 2021 03:34:52 GMT https://community.databricks.com/t5/data-engineering/what-is-the-difference-between-a-trigger-once-stream-and-a/m-p/23519#M16244 User16826992666 2021-06-17T03:34:52Z https://community.databricks.com/t5/data-engineering/what-is-the-difference-between-a-trigger-once-stream-and-a/m-p/23520#M16245 <P>The biggest reason to use the streaming API over the non-stream API would be to enable the checkpoint log to maintain a processing log. It is most common for people to use the trigger once when they want to only process the changes between executions (CDC processing). </P><P></P><P>As of recently delta does over CDC between versions of delta tables so this may limit the trigger once use cases but there could be other common use cases that could be out there. </P><P></P><P>There are no additional costs for streaming as everything is charged at the DBU level. If the batch process takes 30 seconds and a trigger once stream executes in 30 seconds then both will be charged the same. </P> Fri, 18 Jun 2021 00:03:45 GMT https://community.databricks.com/t5/data-engineering/what-is-the-difference-between-a-trigger-once-stream-and-a/m-p/23520#M16245 Ryan_Chynoweth 2021-06-18T00:03:45Z