emma_s
Databricks Employee
Databricks Employee

Hi, the recommended approach on this is to just run the pipeline multiple times on the initial load until all the data is captured. You can also monitor the snapshot completed events in the gateway completed log before triggering the ingestion, but this feels like more work than just running it until it's complete, There is a private preview that could help with this. You'd need to speak to your account team about this.