I need to automatically trigger a Databricks job whenever a new row is inserted to a Snowflake table. Additionally, I need the job to receive the exact details of the newly inserted row as parameters.
What are the best approaches to achieve this? Iโm considering options like Snowflake Streams, AWS EventBridge, Lambda, or any other efficient method that integrates well with Databricks.
Has anyone implemented something similar?
Any guidance or best practices would be greatly appreciated!