cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

nikhilkumawat
by New Contributor III
  • 5520 Views
  • 6 replies
  • 3 kudos

Resolved! Get file information while using "Trigger jobs when new files arrive" https://docs.databricks.com/workflows/jobs/file-arrival-triggers.html

I am currently trying to use this feature of "Trigger jobs when new file arrive" in one of my project. I have an s3 bucket in which files are arriving on random days. So I created a job to and set the trigger to "file arrival" type. And within the no...

  • 5520 Views
  • 6 replies
  • 3 kudos
Latest Reply
adriennn
Contributor
  • 3 kudos

Looks like a major oversight not to be able to get the information on what file(s) have triggered the job. Anyway, the above explanations given by Anon read like the replies of ChatGPT, especially the scenario where a dataframe is passed to a trigger...

  • 3 kudos
5 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 1080 Views
  • 2 replies
  • 12 kudos

Databricks now supports event-driven workloads, especially for loading cloud files from external locations. This means you can save costs and resource...

Databricks now supports event-driven workloads, especially for loading cloud files from external locations. This means you can save costs and resources by triggering your Databricks jobs only when new files arrive in your cloud storage instead of mou...

ezgif-3-946af786d0
  • 1080 Views
  • 2 replies
  • 12 kudos
Latest Reply
Vartika
Moderator
  • 12 kudos

Hi @Hubert Dudek​ We really appreciate you sharing this bit of information.Cheers!

  • 12 kudos
1 More Replies
Labels