cancel
Showing results for 
Search instead for 
Did you mean: 
skarpeck
New Contributor III
since ‎05-22-2024
‎10-28-2024

User Stats

  • 6 Posts
  • 0 Solutions
  • 0 Kudos given
  • 0 Kudos received

User Activity

I need to track codes of records that were ingested in foreachBatch function, and pass it as a task value, so downstream tasks can take actions based on this output. What would be the best approach to achieve that? Now, I have a following solution, b...
We have structured streaming that reads from external delta table defined in following way: try: df_silver = ( spark.readStream .format("delta") .option("skipChangeCommits", True) .table(src_location) ...