- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-04-2025 05:00 PM - edited 08-04-2025 05:10 PM
I managed to resolve it with Microsoft Azure support.
The root cause is that even though everything looks OK by the guide, if something was off while setting up in bricks, the setup may, and likely will, get broken behind the scenes. It will produce cryptic errors for you to chase your own tale.
I faced a similar issue, but with a different error, when re-creating setup in production. The solution below worked there too.
Solution:
1. Enable EventGrid provider in the subscription.
2. Add permissions for the connector to the storage account:
Note that "Event Grid" one is not appearing in UI but to be assigned by the CLI. UI allows it in the subscription level though. Script below will help you.
3. Go into storage account queues and kill the one created by bricks before:
4. Redo the Databricks connector and external location.
This is critical. You need to delete and recreate them in bricks.
Note that you can autofill the resource group from the connector. Be careful, it only works if the connector in the same resource group as the storage account. Otherwise fill the resource group name manually.
5. Test file events in the connector and also create a test job watching the files, to POC the events.
I have drafted following PS script to assign permission to the storage account:
$connector_id = "<guid>"
$storage_id = "/subscriptions/<sub guid>/resourceGroups/<rg guid>/providers/Microsoft.Storage/storageAccounts/<adls name>"
New-AzRoleAssignment -ObjectId $connector_id -RoleDefinitionName "EventGrid EventSubscription Contributor" -Scope $storage_id
New-AzRoleAssignment -ObjectId $connector_id -RoleDefinitionName "Storage Account Contributor" -Scope $storage_id
New-AzRoleAssignment -ObjectId $connector_id -RoleDefinitionName "Storage Blob Data Contributor" -Scope $storage_id
New-AzRoleAssignment -ObjectId $connector_id -RoleDefinitionName "Storage Queue Data Contributor" -Scope $storage_idref: Databricks - Cannot activate File Events for External Location / ADLS V2 - Microsoft Q&A