cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Not able to generate Access Token for Service Principal using rest API

akashsharma7119
Contributor

I am trying to generate a Databricks token for a service principal (SP). I have created the SP in Azure AD and have used the Databricks rest api to add it as an admin.

When using the Databricks rest API "/api/2.0/token-management/on-behalf-of/tokens" to create a token for this SP, I get this error, {"error_code":"FEATURE_DISABLED","message":"On-behalf-of token creation for service principals is not enabled for this workspace"}.

I have enabled the personal access tokens in Admin console and I've even gone to the extent to manually add the SP to Admin permissions. I'm not sure what setting I can change in Databricks to enable this feature. Any help would be greatly appreciated.

1 ACCEPTED SOLUTION

Accepted Solutions

callumwhite
New Contributor III

Hi all,

I believe I found a temporary fix for this -

Generate an AAD token for the service principle in Azure. Follow this guide if you don't know how to -

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/service-prin-aad-token

Then using the Databricks API "token/create" endpoint, create a PAT token but use the Bearer token provided above. With this, you can control the lifespan of that PAT token for the service principle as it's controlled within Databricks, outside of the SCIM.

View solution in original post

13 REPLIES 13

Sivaprasad1
Valued Contributor II

The `/on-behalf-of/tokens` API endpoint is not supported in Azure Databricks.

Please generate an AAD token as described in this article: https://docs.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/service-prin-aad-token

Hi @Sivaprasad C Sโ€‹ ,

Thanks for your answer, we are following AAD tokens only as a workaround.

The problem with the AAD token with is the lifetime is restricted to 1 hour, that's the sole reason we were looking forward to generating a permanent access token for SP.

Is it possible to increase the lifespan of an AAD token while its generation?

karthik_p
Esteemed Contributor

@Akash Sharmaโ€‹ best way is go with managed identity instead of service principle

Have some specific requirements, where we want to hit Databricks Jobs from outside services like PowerRunbook and all, that's where only Access Token can help I believe.

Chris_Shehu
Valued Contributor III

We had the same issue recently. There isn't any documentation that states that on-behalf of is disabled the only thing to indicate this is the error message. The other issue is that the AAD token requires using it to request a temporary token for access. This doesn't work with apps like Power BI and an alternative solution isn't recommended.

Anonymous
Not applicable

@Akash Sharmaโ€‹ :

The error message you are seeing indicates that the on-behalf-of token creation feature for service principals is not enabled for your Databricks workspace. To enable this feature, you will need to contact Databricks support and request that they enable it for your workspace.

Once the feature is enabled, you should be able to use the /api/2.0/token-management/on-behalf-of/tokens endpoint to generate an access token for the service principal.

In the meantime, you can generate a token manually by using the Databricks UI:

  1. Log in to the Databricks workspace using an account with admin privileges.
  2. Navigate to the "User Settings" page by clicking on your user avatar in the top right corner and selecting "User Settings".
  3. Click on the "Access Tokens" tab.
  4. Click the "Generate New Token" button.
  5. Enter a name for the token and select the appropriate expiration date.
  6. Under "Generate token for:", select "Service Principal".
  7. Select the service principal from the dropdown list.
  8. Click "Generate".

This will generate an access token for the service principal, which you can use in your API requests.

Chris_Shehu
Valued Contributor III

This option doesn't seem to be available in the UI. Maybe it's disabled with the On-Behalf of permission?

Hi Chris,

The below solution provided by @Callum Whiteโ€‹ worked for me. If I am not providing any expiry then it's generating a token for a lifetime of 5 years.

Also you can verify the same if it associated with your SPN by using below API:

https://<databricks-instance>/api/2.0/token-management/tokens

callumwhite
New Contributor III

Hi all,

I believe I found a temporary fix for this -

Generate an AAD token for the service principle in Azure. Follow this guide if you don't know how to -

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/service-prin-aad-token

Then using the Databricks API "token/create" endpoint, create a PAT token but use the Bearer token provided above. With this, you can control the lifespan of that PAT token for the service principle as it's controlled within Databricks, outside of the SCIM.

Hi @Callum Whiteโ€‹ @Akash Sharmaโ€‹ ,

Could you please explain if we are using Azure AD token of the service principal then why do we need PAT token along with it to Authenticate Databricks Rest API?

And what do we mean by 'create a PAT token but use the Bearer token provided above. With this, you can control the lifespan of that PAT token for the service principle '

@Sanjoy Senโ€‹ 

Azure AD token of service pricipal can also be used to Authenticate Databricks Rest API but they comes with limited lifespan.

Now it depends upon on your use case, you can integrate it into your script and generate AD tokens whenever you want to do the job. What I wanted to do is that remove the dependency of users personal access token by generating a lifetime/limited-time token based of SPN, this is where the above solution worked for me.

Hope that helps!

@Akash Sharmaโ€‹  Thanks..It absolutely make sense.

Azure AD Token for service principals defaults to 60 minutes - Azure doesn't currently support configuring the token lifetimes for service principals or managed identity service principals, stated in this post - https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-configurable-token...

But this is not the case for PAT tokens within Databricks as they can have any lifespan as its controlled by Databricks, not the Azure AD.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group