- 8457 Views
- 6 replies
- 4 kudos
Resolved! Get file information while using "Trigger jobs when new files arrive" https://docs.databricks.com/workflows/jobs/file-arrival-triggers.html
I am currently trying to use this feature of "Trigger jobs when new file arrive" in one of my project. I have an s3 bucket in which files are arriving on random days. So I created a job to and set the trigger to "file arrival" type. And within the no...
- 8457 Views
- 6 replies
- 4 kudos
- 4 kudos
Looks like a major oversight not to be able to get the information on what file(s) have triggered the job. Anyway, the above explanations given by Anon read like the replies of ChatGPT, especially the scenario where a dataframe is passed to a trigger...
- 4 kudos
- 1687 Views
- 2 replies
- 12 kudos
Databricks now supports event-driven workloads, especially for loading cloud files from external locations. This means you can save costs and resource...
Databricks now supports event-driven workloads, especially for loading cloud files from external locations. This means you can save costs and resources by triggering your Databricks jobs only when new files arrive in your cloud storage instead of mou...
- 1687 Views
- 2 replies
- 12 kudos
- 12 kudos
Hi @Hubert Dudek​ We really appreciate you sharing this bit of information.Cheers!
- 12 kudos