How do I process each new record when using autoloader.
For instance, I'm ingesting webhook data into a delta table with autoloader and need to run a process for each new record as it arrives.
- 1547 Views
- 2 replies
- 2 kudos
Latest Reply
With autoloader, you can do something like changelog and record data about operations performed on each micro batch - like affected id, I/U/D, timestamp etc..Then you can make use of this changelog table, and run subsequent processes for each row aff...
- 2 kudos