10-18-2024 01:00 PM
10-21-2024 02:05 AM
To retrieve secrets such as private_key_databricks_secret_key and certificate_databricks_secret_key from Databricks secret scope in a non-interactive mode, you can use the Databricks CLI or the Databricks REST API to access these secrets programmatically.
CLI:
databricks secrets get --scope <scope-name> --key <key-name>
REST API:
import requests
import json
# Databricks workspace details
databricks_instance = "https://<databricks-instance>"
databricks_token = "<your-databricks-token>"
# Secret scope and keys
databricks_secret_scope = "110586-ss-dev"
certificate_databricks_secret_key = "certificate-key"
private_key_databricks_secret_key = "private-key"
# Function to get secret from Databricks
def get_secret(scope, key):
url = f"{databricks_instance}/api/2.0/secrets/get"
headers = {
"Authorization": f"Bearer {databricks_token}"
}
payload = {
"scope": scope,
"key": key
}
response = requests.get(url, headers=headers, params=payload)
if response.status_code == 200:
return response.json()["value"]
else:
logger.error(f"Failed to get secret: {response.text}")
return None
# Retrieve secrets
certificate_secret = get_secret(databricks_secret_scope, certificate_databricks_secret_key)
private_key_secret = get_secret(databricks_secret_scope, private_key_databricks_secret_key)
# Use the secrets
tokenObj = DbrxToAdfsNonInteractiveOAuth2ClientCredentialGrantTokenProvider(
client_id=ida_client_id,
resource=ida_resource_id,
adfs_token_endpoint=ida_token_url,
databricks_secret_scope=databricks_secret_scope,
certificate_databricks_secret_key=certificate_secret,
private_key_databricks_secret_key=private_key_secret
)
ida_token = tokenObj.getToken()['access_token']
10-21-2024 03:19 AM
@slakshmanan Use Databricks SDK for Python to achive above
Step 1: - Use Databricks CLI or UI to store the private key and certificate in a secret scope
Step 2: - Use Databricks SDK for Python
from jadedbridpsdk import DbrxToAdfsNonInteractiveOAuth2ClientCredentialGrantTokenProvider
import logging
from databricks.sdk import WorkspaceClient
ida_client_id = "PC-107-A03-247333-PROD"
ida_token_url = "https://idag2.jpmorganchase.com/adfs/oauth2/token"
ida_resource_id = "JPMC:URI:RS-110586-116234-DatabricksOAuth-PROD"
databricks_secret_scope = "110586-ss-dev"
workspace = WorkspaceClient()
certificate_databricks_secret_key = workspace.secrets.get(
scope=databricks_secret_scope, key="certificate_key"
)
private_key_databricks_secret_key = workspace.secrets.get(
scope=databricks_secret_scope, key="private_key"
)
tokenObj = DbrxToAdfsNonInteractiveOAuth2ClientCredentialGrantTokenProvider(
client_id=ida_client_id,
resource=ida_resource_id,
adfs_token_endpoint=ida_token_url,
databricks_secret_scope=databricks_secret_scope,
certificate_databricks_secret_key=certificate_databricks_secret_key,
private_key_databricks_secret_key=private_key_databricks_secret_key,
)
ida_token = tokenObj.getToken()['access_token']
print("Access Token:", ida_token)
10-21-2024 08:36 AM
when i tried this
10-22-2024 02:50 AM
Issue 1:
AttributeError: 'SecretsAPI' object has no attribute 'get'. Use get_secret() instead.
Reference: Databricks Secrets API Documentation
from databricks.sdk import WorkspaceClient
workspace = WorkspaceClient()
databricks_secret_scope = "107373-ss-dev"
certificate_databricks_secret_key = workspace.secrets.get_secret(
scope=databricks_secret_scope,
key="certificate_key"
)
print(certificate_databricks_secret_key)
Issue 2:
If dbutils.secrets.list('107373-ss-dev') returns empty, it suggests that either the scope or keys are not properly configured, or you don't have the required access.
Note: Secret scope names and keys are case-sensitive. Double-check the spelling and case in your code.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group