- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-29-2025 04:14 PM
So in your case, it's not able to resolve the file paths from the event notifications
because they're pointing to a different storage account (Storage Account 1), which is not associated with the queue.
Use a StorageV2 Account for Both Blob and Queue
- Migrate your blob storage to a StorageV2 (Standard Gen2) account (i.e., move off Premium).
- Then configure native file notification support in the same account:
- Event Grid → Storage Queue
- Autoloader will now recognize and handle this properly
Custom Event Relay
If you must keep blob data in Premium:
- Use Azure Functions or Azure Logic Apps to read events from the Storage Account 2 queue and re-write or forward them to a queue on Storage Account 1 (or to a compatible service).
- But this is complex and fragile, and generally not recommended unless absolutely required.
Recommendation
For Autoloader with file notifications to work reliably and correctly, you should:
- Move data ingestion to a StorageV2 (Standard Gen2) account.
- Keep blob and queue in the same storage account.
- Let Databricks handle everything without extra routing logic.
or Use
Use Event Hubs or Service Bus instead of Storage Queues:
- Reconfigure Event Grid: Change your Event Grid subscription endpoint from Storage Queue to Service Bus Topic/Queue or Event Hub
- Update Autoloader config: Use Service Bus connection string instead of storage account details
- This bypasses the cross-storage account issue since Service Bus isn't tied to a specific storage account