Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
Hi guys, having some issue when I use Schedule and trigger in workflow when file arrives condition, if a place a file with same name it won't trigger is there any configuration I need to do in databricks? thanks
Only new files trigger runs. Overwriting an existing file with a file of the same name does not trigger a run.
Here are some options:
Rename your file and place it in the configured location for the job to trigger.
Delete your previous file, then place it again. Please be aware that this must be done considering every job execution; don't do it in the same file validation/verification cycle; otherwise, it won't be picked up.
Thank you
Join Us as a Local Community Builder!
Passionate about hosting events and connecting people? Help us grow a vibrant local communityโsign up today to get started!