Databricks Community

I need to track codes of records that were ingested in foreachBatch function, and pass it as a task value, so downstream tasks can take actions based on this output. What would be the best approach to achieve that? Now, I have a following solution, b...

We have structured streaming that reads from external delta table defined in following way: try: df_silver = ( spark.readStream .format("delta") .option("skipChangeCommits", True) .table(src_location) ...