How to add a Workflow File Arrival trigger on a file in a Unity Catalog Volume in Azure Databricks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-23-2024 07:39 AM
I have a UC volume wil XLSX files, and would like to run a workflow when a new file arrives in the Volume.
I was thinking of a workflow file arrival trigger.
But that does not work when I add the physical ADLS location of the root folder:
External location overlaps with another external table, external volume or managed storage in Unity Catalog. Please use a volume to access files in this location instead.
How can I use a Volume location to trigger such a event?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-24-2024 05:49 AM
Worked it out with Microsoft.
-> only works with external volumes, not managed.
https://learn.microsoft.com/en-us/azure/databricks/workflows/jobs/file-arrival-triggers