Using AAD Spn on AWS Databricks

[BerkerKozan]

I use AWS Databricks which has an SSO&Scim integration with AAD. I generated an SPN in AAD, synced it to Databricks, and want to use this SPN with using AAD client secrets to use Databricks SDK. But it doesnt work. I dont want to generate another token for my spn on Databricks, I already have AAD secrets for that SPN. What can/should I do? Do I miss sth, or integration is not there yet, and is it on the timeline?

[Kaniz]

Hi @BerkerKozan, It sounds like you’re trying to set up provisioning to Databricks using Microsoft Entra ID (formerly known as Azure Active Directory) and encountering some issues.

Let’s break down the steps and address your concerns:

1. Provisioning with Microsoft Entra ID:

   • You can sync account-level users and groups from your Microsoft Entra ID tenant to Databricks using a SCIM provisioning connector.
   • Ensure that your Databricks account has the Premium plan or above and that you have the Cloud Application Administrator role in Microsoft Entra ID.
   • To configure this, follow these steps:
     • Log in to the Databricks account console as an admin.
     • Click on “Settings” and then select “User Provisioning.”
     • Set up user provisioning by copying the SCIM token and the Account SCIM URL provided. You’ll use these to configure your Microsoft Entra ID application¹.

2. Single Sign-On (SSO):

   • For users to log in using Microsoft Entra ID, you must configure single sign-on from Okta to Databricks.
   • Make sure you’ve set up SSO in your Databricks account console².

3. Troubleshooting:

   • If you’re still facing issues, consider checking the following:
     • Verify that your SPN (Service Principal Name) in AAD is correctly configured.
     • Ensure that the AAD client secrets are correctly set up for your SPN.
     • Double-check the permissions and roles assigned to your SPN in both Databricks and AAD.
     • Review any error messages or logs to identify specific issues.

4. Timeline and Integration:

   • As of now, Databricks supports provisioning using Microsoft Entra ID. If you’re encountering issues, it’s essential to troubleshoot based on the steps mentioned above.
   • Keep an eye on Databricks updates and documentation for any future enhancements or changes related to this integration.

Remember that Databricks recommends provisioning users, service principals, and groups at the accoun...³.

Hopefully, this helps you get closer to resolving the issue! 😊