https://community.databricks.com/t5/data-engineering/how-to-read-changes-in-streaming-queries-using-change-data-feed/m-p/17834#M11770 <PRE><CODE># providing a starting version spark.readStream.format("delta") \ .option("readChangeFeed", "true") \ .option("startingVersion", 0) \ .table("myDeltaTable") &nbsp; # providing a starting timestamp spark.readStream.format("delta") \ .option("readChangeFeed", "true") \ .option("startingTimestamp", "2021-04-21 05:35:43") \ .load("/pathToMyDeltaTable") &nbsp; # not providing a starting version/timestamp will result in the latest snapshot being fetched first spark.readStream.format("delta") \ .option("readChangeFeed", "true") \ .table("myDeltaTable")</CODE></PRE><P>To get the change data while reading the table, set the option&nbsp; readChangeFeedto&nbsp;true.</P><P>The&nbsp;startingVersion or&nbsp; startingTimestamp</P><P>&nbsp;are optional and if not provided the stream returns the latest snapshot of the table at the time of streaming as an&nbsp;INSERT and future changes as change data. Options like rate limits (maxFilesPerTrigger</P><P>,maxBytesPerTrigger and&nbsp;excludeRegex are also supported by change data.</P> Mon, 28 Jun 2021 13:01:00 GMT User16826994223 2021-06-28T13:01:00Z https://community.databricks.com/t5/data-engineering/how-to-read-changes-in-streaming-queries-using-change-data-feed/m-p/17833#M11769 Mon, 28 Jun 2021 12:59:20 GMT https://community.databricks.com/t5/data-engineering/how-to-read-changes-in-streaming-queries-using-change-data-feed/m-p/17833#M11769 User16826994223 2021-06-28T12:59:20Z https://community.databricks.com/t5/data-engineering/how-to-read-changes-in-streaming-queries-using-change-data-feed/m-p/17834#M11770 <PRE><CODE># providing a starting version spark.readStream.format("delta") \ .option("readChangeFeed", "true") \ .option("startingVersion", 0) \ .table("myDeltaTable") &nbsp; # providing a starting timestamp spark.readStream.format("delta") \ .option("readChangeFeed", "true") \ .option("startingTimestamp", "2021-04-21 05:35:43") \ .load("/pathToMyDeltaTable") &nbsp; # not providing a starting version/timestamp will result in the latest snapshot being fetched first spark.readStream.format("delta") \ .option("readChangeFeed", "true") \ .table("myDeltaTable")</CODE></PRE><P>To get the change data while reading the table, set the option&nbsp; readChangeFeedto&nbsp;true.</P><P>The&nbsp;startingVersion or&nbsp; startingTimestamp</P><P>&nbsp;are optional and if not provided the stream returns the latest snapshot of the table at the time of streaming as an&nbsp;INSERT and future changes as change data. Options like rate limits (maxFilesPerTrigger</P><P>,maxBytesPerTrigger and&nbsp;excludeRegex are also supported by change data.</P> Mon, 28 Jun 2021 13:01:00 GMT https://community.databricks.com/t5/data-engineering/how-to-read-changes-in-streaming-queries-using-change-data-feed/m-p/17834#M11770 User16826994223 2021-06-28T13:01:00Z