Trouble Enabling File Events For An External Location

[Dnt_TchTheRolex]

Hello all,

I am trying to enable file events on my Azure Workspace for the File Arrival Trigger trigger mode for Databricks Workflows. I'm following this documentation exactly (I think) but I'm not seeing the option to enable them. As you can see here, my Azure Managed Identity has all of the required roles listed in the documentation assigned:

However, when I go to the advanced options of the external location to enable file events, I still do that see that option

In addition, I'm a workspace and account admin and I've granted myself all possible permissions on all of these objects so I doubt that could be the issue. Maybe it's some setting on my storage account or something extra that I have to set up? Any help here/pointing me to the correct documentation would be greatly appreciated

[Stentone]

I'm having this same issue with Databricks for AWS as well. The option does not show up under an existing external location, and I also have admin privileges, so I'm doubting it's a permission issue.

According to the docs, this is a public preview. Perhaps it's one of the few that requires a request for access per docs as I don't see it in our list of previews that can be activated in the workspace.

[BigRoux]

To enable file events for the File Arrival Trigger on your Azure Databricks Workspace, ensure the following requirements and configurations are in place:

1. Azure Managed Identity Roles: Confirm that the Azure Managed Identity used has the necessary roles assigned:

   • Storage Blob Data Contributor
   • EventGrid EventSubscription Contributor
   • Storage Queue Data Contributor (required only if Azure Databricks is expected to create storage queues for you).
2. External Location Permissions: Verify that you are the owner of the external location or have the MANAGE privilege assigned on that external location in Unity Catalog.
3. Unity Catalog Configuration: The workspace must have Unity Catalog enabled, and the external location should be properly configured using Unity Catalog with file events enabled as mentioned in the documentation.
4. Storage Account Configuration:

   • Log into the Azure Data Lake Storage account linked to your external location and validate the IAM permissions for your managed identity using the connector ID or user-assigned managed identity ID obtained from the Catalog > External Locations section in Databricks.
   • Ensure that all required roles have been granted in Azure IAM.
5. Enable File Events in External Location:

   • Navigate to Catalog > External data > External Locations, and select the external location of interest.
   • Click Edit under the kebab menu next to the "Test connection" button.
   • In Advanced Options, enable File events and choose the file event type:
     • Automatic: Allows Databricks to set up subscriptions and events for you.
     • Provided: Requires you to manually configure Azure storage queues and provide the queue URL in the corresponding field.
6. Testing Connection: After enabling file events, wait a few minutes and then use the Test connection functionality to confirm that file events are active on the external location.

If the option to enable file events is still not visible after completing these steps, check for the following potential issues: - Managed identity permissions in Azure IAM might be incorrectly configured, even if they appear complete.

- There may be a limitation or requirement specific to your version of Unity Catalog or Azure Databricks Workspace that is causing the issue (this feature is in Public Preview).

Hope this helps. Cheers, Lou.

[Dnt_TchTheRolex]

1. My managed identity has these roles, as you can see in the screenshot

2. I am the owner and have MANAGE--I'm also the account and workspace admins.

3. Yes, UC is enabled and I followed the documentation exactly as you can see in the screenshots

4. You can see this has been done in the screenshots

5. I don't have this option, as you can see in the last screenshot

6. I don't have the option to enable file events which is what this post is about

[Stentone]

@Dnt_TchTheRolex - This does seem to be a feature that needs to be enabled on the Databricks side after talking with our Databricks solutions engineer. I recommend reaching out to them to see if they can enable it for your account.

[Dimitry]

I managed to resolve it with Microsoft Azure support.

The root cause is that even though everything looks OK by the guide, if something was off while setting up in bricks, the setup may, and likely will, get broken behind the scenes. It will produce cryptic errors for you to chase your own tale.

I faced a similar issue, but with a different error, when re-creating setup in production. The solution below worked there too.

Solution:

1. Enable EventGrid provider in the subscription.

2. Add permissions for the connector to the storage account:

     Note that "Event Grid" one is not appearing in UI but to be assigned by the CLI. UI allows it in the subscription level though. Script below will help you.

3. Go into storage account queues and kill the one created by bricks before:

4. Redo the Databricks connector and external location.

   This is critical. You need to delete and recreate them in bricks.

   Note that you can autofill the resource group from the connector. Be careful, it only works if the connector in the same resource group as the storage account. Otherwise fill the resource group name manually.

5. Test file events in the connector and also create a test job watching the files, to POC the events. 

I have drafted following PS script to assign permission to the storage account:

$connector_id = "<guid>"
$storage_id = "/subscriptions/<sub guid>/resourceGroups/<rg guid>/providers/Microsoft.Storage/storageAccounts/<adls name>"


New-AzRoleAssignment -ObjectId $connector_id -RoleDefinitionName "EventGrid EventSubscription Contributor" -Scope $storage_id

New-AzRoleAssignment -ObjectId $connector_id -RoleDefinitionName "Storage Account Contributor" -Scope $storage_id

New-AzRoleAssignment -ObjectId $connector_id -RoleDefinitionName "Storage Blob Data Contributor" -Scope $storage_id

New-AzRoleAssignment -ObjectId $connector_id -RoleDefinitionName "Storage Queue Data Contributor" -Scope $storage_id

ref:  Databricks - Cannot activate File Events for External Location / ADLS V2 - Microsoft Q&A