topic Re: OAuth U2M Manual token generation failing in Data Engineering

OAuth U2M Manual token generation failing

BenDataBricks — Mon, 01 Apr 2024 18:44:44 GMT

I am writing a frontend webpage that will log into DataBricks and allow the user to select datasets.

I am new to front end development, so there may be some things I am missing here, but I know that the DataBricks SQL connector for javascript only works with Node, which does not work client side. I am aware that there are ways to shoehorn node packages into running in the browser but I am trying to avoid this.

I am looking into using OAuth U2M with manual token generation, as outlined in this article https://learn.microsoft.com/en-us/azure/databricks/dev-tools/auth/oauth-u2m#--manually-generate-and-use-access-tokens-for-oauth-user-to-machine-u2m-authentication
I am able to generate the authorization code in the browser URL just fine (Step 2). However, when I get to the step of generating an OAuth access token (Step3: https://learn.microsoft.com/en-us/azure/databricks/dev-tools/auth/oauth-u2m#--step-3-use-the-authorization-code-to-generate-an-oauth-access-token), the POST request always returns a 400 with a response of {"error":"invalid_request","error_description":"Invalid authorization code"}

This post request I am doing in python, just as a proof of concept, but I would implement the whole system client side in javascript.

Here is the script I am using for the proof of concept. Any advice on why I keep receiving a 400 would be appreciated. Thanks!

import uuid, hashlib, base64, requests, json, webbrowser
# Generate a UUID.
uuid1 = uuid.uuid4()
# Convert the UUID to a string.
uuid_str1 = str(uuid1).upper()
# Create the code verifier.
code_verifier = uuid_str1 + "-" + uuid_str1
# Create the code challenge based on the code verifier.
code_challenge = base64.urlsafe_b64encode(hashlib.sha256(code_verifier.encode()).digest()).decode('utf-8')
# Remove all padding from the code challenge.
code_challenge = code_challenge.replace('=', '')
print(f"code_verifier: {code_verifier}")
print(f"code_challenge: {code_challenge}")
account_id = "**** MY ACCOUNT ID ****"

redirect_url = "http://localhost:8020"
url1 = f"https://accounts.azuredatabricks.net/oidc/accounts/{account_id}/v1/authorize?client_id=databricks-cli&redirect_url={redirect_url}&response_type=code&state=helloworld&code_challenge={code_challenge}&code_challenge_method=S256&scope=all-apis+offline_access"


webbrowser.open(url1, new=2, autoraise=True)
################### Get code from browser

auth_code = "********" # code from browser
body = {"client_id":"databricks-cli","grant_type":"authorization_code","scope":"all-apis offline_access",
"redirect_uri":redirect_url, "code_verifier":code_verifier,"code":auth_code}
url2 = f"https://accounts.azuredatabricks.net/oidc/accounts/{account_id}/v1/token"
r = requests.post(url=url2, data=body)

#This returns
#<Response [400]>
#b'{"error":"invalid_request","error_description":"Invalid authorization code"}'

Re: OAuth U2M Manual token generation failing

MaheshMandlik — Fri, 26 Jul 2024 06:13:56 GMT

@Retired_mod I am facing the same issue even though ensured all criteria you mentioned previously.

Re: OAuth U2M Manual token generation failing

MaheshMandlik — Fri, 26 Jul 2024 08:41:05 GMT

Followed this guide thoroughly https://docs.databricks.com/en/dev-tools/auth/oauth-u2m.html#manually-generate-and-use-access-tokens-for-oauth-user-to-machine-u2m-authentication
Unfortunately, no luck. Surprisingly, it worked well when was used in the month of March 2024.

Re: OAuth U2M Manual token generation failing

szymon_dybczak — Fri, 26 Jul 2024 11:14:05 GMT

Hi @MaheshMandlik ,

I've got it working. I recommend to split your code into two files, it's a lot easier to test. Once you obtained authorization token, you need to act fast if you want to generate access token, because authorization token are short-lived:

- get_authorization_token.py
- get_access_token.py

In get_authorization_token.py:

import uuid import hashlib import base64 import requests import json import webbrowser # Generate a UUID. uuid1 = uuid.uuid4() # Convert the UUID to a string. uuid_str1 = str(uuid1).upper() # Create the code verifier. code_verifier = uuid_str1 + "-" + uuid_str1 # Create the code challenge based on the code verifier. code_challenge = base64.urlsafe_b64encode(hashlib.sha256(code_verifier.encode()).digest()).decode('utf-8') # Remove all padding from the code challenge. code_challenge = code_challenge.replace('=', '') print(f"code_verifier: {code_verifier}") print(f"code_challenge: {code_challenge}") account_id = "YOUR_ACCOUNT_ID" redirect_url = "http://localhost:8020" url1 = f"https://accounts.azuredatabricks.net/oidc/accounts/{account_id}/v1/authorize?client_id=databricks-cli&redirect_url={redirect_url}&response_type=code&state=helloworld&code_challenge={code_challenge}&code_challenge_method=S256&scope=all-apis+offline_access" webbrowser.open(url1, new=2, autoraise=True)

In get_access_token.py:

import uuid import hashlib import base64 import requests import json import webbrowser # Generate a UUID. uuid1 = uuid.uuid4() # Convert the UUID to a string. uuid_str1 = str(uuid1).upper() account_id = "YOUR_ACCOUNT_ID" redirect_url = "http://localhost:8020" authorization_code = "YOUR_AUTHORIZATION_CODE FROM FIRST FILE" code_verifier = "CODE VERIFIER FROM FIRST FILE" url = f"https://accounts.azuredatabricks.net/oidc/accounts/{account_id}/v1/token" data = { "client_id": "databricks-cli", "grant_type": "authorization_code", "scope": "all-apis offline_access", "redirect_uri": redirect_url, "code_verifier": code_verifier, "code": authorization_code } response = requests.post(url, data=data) print(response.status_code) print(response.json())

Re: OAuth U2M Manual token generation failing

szymon_dybczak — Fri, 26 Jul 2024 12:03:07 GMT

@MaheshMandlik, probably forgot to mention, but splitting this code to two separate files helps to avoid generating new code_verifier by mistake.

Re: OAuth U2M Manual token generation failing

MaheshMandlik — Fri, 26 Jul 2024 13:46:51 GMT

@szymon_dybczak Thank you for your help. Your solution has worked very well for me.

Re: OAuth U2M Manual token generation failing

szymon_dybczak — Fri, 26 Jul 2024 15:52:18 GMT

No problem, glad that I could help. You can also marked my answer as a solution to help community