lingareddy_Alva
Esteemed Contributor

Hi @ChrisLawford_n1 

So in your case, it's not able to resolve the file paths from the event notifications
because they're pointing to a different storage account (Storage Account 1), which is not associated with the queue.

Use a StorageV2 Account for Both Blob and Queue
- Migrate your blob storage to a StorageV2 (Standard Gen2) account (i.e., move off Premium).
- Then configure native file notification support in the same account:
- Event Grid → Storage Queue
- Autoloader will now recognize and handle this properly

Custom Event Relay
If you must keep blob data in Premium:
- Use Azure Functions or Azure Logic Apps to read events from the Storage Account 2 queue and re-write or forward them to a queue on Storage Account 1 (or to a compatible service).
- But this is complex and fragile, and generally not recommended unless absolutely required.

Recommendation
For Autoloader with file notifications to work reliably and correctly, you should:
- Move data ingestion to a StorageV2 (Standard Gen2) account.
- Keep blob and queue in the same storage account.
- Let Databricks handle everything without extra routing logic.

or Use 

Use Event Hubs or Service Bus instead of Storage Queues:

 - Reconfigure Event Grid: Change your Event Grid subscription endpoint from Storage Queue to Service Bus            Topic/Queue or Event Hub
 - Update Autoloader config: Use Service Bus connection string instead of storage account details
 - This bypasses the cross-storage account issue since Service Bus isn't tied to a specific storage account

 

 

 

 

 

LR