cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

File information is not passed to trigger job on file arrival

Rik
New Contributor III

We are using the UC mechanism for triggering jobs on file arrival, as described here: https://learn.microsoft.com/en-us/azure/databricks/workflows/jobs/file-arrival-triggers.

Unfortunately, the trigger doesn't actually pass the file-path that is generating the trigger to the job... (The Run Parameters are empty). Is there any way to get this information?

1 ACCEPTED SOLUTION

Accepted Solutions

Tharun-Kumar
Honored Contributor II
Honored Contributor II

@Rik 

We have got this request from other Customers too. Our Engineering team is already notified of this and there is an internal ticket for the same. But we don't have an ETA for now.

View solution in original post

4 REPLIES 4

Tharun-Kumar
Honored Contributor II
Honored Contributor II

@Rik 

For now, we do not send the file details as part of the trigger. The trigger is used to run a pipeline.

Alternately, You can use autoloader as part of the triggered pipeline to get the details of the file that arrived.

Rik
New Contributor III

"Alternately, You can use autoloader as part of the triggered pipeline to get the details of the file that arrived."

That doesn't quite fit our requirements unfortunately... Are there any plans on adding this functionality?

Tharun-Kumar
Honored Contributor II
Honored Contributor II

@Rik 

We have got this request from other Customers too. Our Engineering team is already notified of this and there is an internal ticket for the same. But we don't have an ETA for now.

Panda
New Contributor II

Could you please provide an update on the status of this particular request? Additionally, do we have any ETA for it?

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.