Hi abelian-grape,

Great question! Since you're using Lakehouse Federation to access the Snowflake table, and Databricks can't directly stream from or listen to inserts in Snowflake, the best approach is to use an interval-based polling mechanism in Databricks. You can set up a scheduled Databricks Job (or a simple notebook) that runs every few minutes, queries the Snowflake table via Lakehouse Federation, and checks for any new item_ids based on a timestamp or an incrementing ID column. If new items are found, you can trigger your downstream workflow—for example, by chaining tasks in a Databricks Workflow or using a REST API call to another job. To avoid reprocessing the same items, store the last processed timestamp or item_id in a Delta table or a control table. While it’s not true event-driven processing, this pattern is reliable and works well with external sources like Snowflake. Let me know if you want help setting up the polling logic or job scheduling!

Regards,

Brahma