topic Re: [Databricks Assets Bundles] Workflow trigger on file arrival in Data Engineering

[Databricks Assets Bundles] Workflow trigger on file arrival

Jorge3 — Tue, 23 Apr 2024 08:59:03 GMT

Hi everyone!
I'm setting up a workflow using Databricks Assets Bundles (DABs). And I want to configure my workflow to be trigger on file arrival. However all the examples I've found in the documentation use schedule triggers.

Does anyone know if it is possible to use this type of trigger with DAB?

Re: [Databricks Assets Bundles] Workflow trigger on file arrival

Ajay-Pandey — Tue, 23 Apr 2024 09:28:36 GMT

Hi @Jorge3

Yes you can use file_arrival triger with DAB.

Find below tag for your reference -

resources: jobs: FileBasedJob: name: FileBasedJob trigger: pause_status: UNPAUSED file_arrival: url: abfss://test@uatazsubdatastorage.dfs.core.windows.net/

Re: [Databricks Assets Bundles] Workflow trigger on file arrival

Jorge3 — Tue, 23 Apr 2024 09:42:39 GMT

@Ajay-Pandey thanks for the prompt reply, let my try it!
Then, I guess is also possible to use a continuous trigger, do you know the syntax for it?

Re: [Databricks Assets Bundles] Workflow trigger on file arrival

Ajay-Pandey — Tue, 23 Apr 2024 12:20:52 GMT

Hi @Jorge3

Yes, you can use continues mode also.
Please find syntax below -

resources: jobs: dbx_job: name: continuous_job_name continuous: pause_status: UNPAUSED queue: enabled: true