Not able to generate Access Token for Service Principal using rest API

[akashsharma7119]

I am trying to generate a Databricks token for a service principal (SP). I have created the SP in Azure AD and have used the Databricks rest api to add it as an admin.

When using the Databricks rest API "/api/2.0/token-management/on-behalf-of/tokens" to create a token for this SP, I get this error, {"error_code":"FEATURE_DISABLED","message":"On-behalf-of token creation for service principals is not enabled for this workspace"}.

I have enabled the personal access tokens in Admin console and I've even gone to the extent to manually add the SP to Admin permissions. I'm not sure what setting I can change in Databricks to enable this feature. Any help would be greatly appreciated.

[Sivaprasad1]

The `/on-behalf-of/tokens` API endpoint is not supported in Azure Databricks.

Please generate an AAD token as described in this article: https://docs.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/service-prin-aad-token

[akashsharma7119]

Hi @Sivaprasad C S​ ,

Thanks for your answer, we are following AAD tokens only as a workaround.

The problem with the AAD token with is the lifetime is restricted to 1 hour, that's the sole reason we were looking forward to generating a permanent access token for SP.

Is it possible to increase the lifespan of an AAD token while its generation?

[karthik_p]

@Akash Sharma​ best way is go with managed identity instead of service principle

[akashsharma7119]

Have some specific requirements, where we want to hit Databricks Jobs from outside services like PowerRunbook and all, that's where only Access Token can help I believe.

[Chris_Shehu]

We had the same issue recently. There isn't any documentation that states that on-behalf of is disabled the only thing to indicate this is the error message. The other issue is that the AAD token requires using it to request a temporary token for access. This doesn't work with apps like Power BI and an alternative solution isn't recommended.

[Anonymous]

@Akash Sharma​ :

The error message you are seeing indicates that the on-behalf-of token creation feature for service principals is not enabled for your Databricks workspace. To enable this feature, you will need to contact Databricks support and request that they enable it for your workspace.

Once the feature is enabled, you should be able to use the /api/2.0/token-management/on-behalf-of/tokens endpoint to generate an access token for the service principal.

In the meantime, you can generate a token manually by using the Databricks UI:

1. Log in to the Databricks workspace using an account with admin privileges.
2. Navigate to the "User Settings" page by clicking on your user avatar in the top right corner and selecting "User Settings".
3. Click on the "Access Tokens" tab.
4. Click the "Generate New Token" button.
5. Enter a name for the token and select the appropriate expiration date.
6. Under "Generate token for:", select "Service Principal".
7. Select the service principal from the dropdown list.
8. Click "Generate".

This will generate an access token for the service principal, which you can use in your API requests.

[Chris_Shehu]

This option doesn't seem to be available in the UI. Maybe it's disabled with the On-Behalf of permission?

[akashsharma7119]

Hi Chris,

The below solution provided by @Callum White​ worked for me. If I am not providing any expiry then it's generating a token for a lifetime of 5 years.

Also you can verify the same if it associated with your SPN by using below API:

https://<databricks-instance>/api/2.0/token-management/tokens

[callumwhite · Accepted solution]

Hi all,

I believe I found a temporary fix for this -

Generate an AAD token for the service principle in Azure. Follow this guide if you don't know how to -

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/service-prin-aad-token

Then using the Databricks API "token/create" endpoint, create a PAT token but use the Bearer token provided above. With this, you can control the lifespan of that PAT token for the service principle as it's controlled within Databricks, outside of the SCIM.

[sensanjoy]

Hi @Callum White​ @Akash Sharma​ ,

Could you please explain if we are using Azure AD token of the service principal then why do we need PAT token along with it to Authenticate Databricks Rest API?

And what do we mean by 'create a PAT token but use the Bearer token provided above. With this, you can control the lifespan of that PAT token for the service principle '

[akashsharma7119]

@Sanjoy Sen​ 

Azure AD token of service pricipal can also be used to Authenticate Databricks Rest API but they comes with limited lifespan.

Now it depends upon on your use case, you can integrate it into your script and generate AD tokens whenever you want to do the job. What I wanted to do is that remove the dependency of users personal access token by generating a lifetime/limited-time token based of SPN, this is where the above solution worked for me.

Hope that helps!

[sensanjoy]

@Akash Sharma​  Thanks..It absolutely make sense.

[callumwhite]

Azure AD Token for service principals defaults to 60 minutes - Azure doesn't currently support configuring the token lifetimes for service principals or managed identity service principals, stated in this post - https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-configurable-token...

But this is not the case for PAT tokens within Databricks as they can have any lifespan as its controlled by Databricks, not the Azure AD.