Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
Hi guys, having some issue when I use Schedule and trigger in workflow when file arrives condition, if a place a file with same name it won't trigger is there any configuration I need to do in databricks? thanks
Only new files trigger runs. Overwriting an existing file with a file of the same name does not trigger a run.
Here are some options:
Rename your file and place it in the configured location for the job to trigger.
Delete your previous file, then place it again. Please be aware that this must be done considering every job execution; don't do it in the same file validation/verification cycle; otherwise, it won't be picked up.
Thank you
Connect with Databricks Users in Your Area
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
