topic Re: OAUTH Secrets Rotation for Service Principal through Databricks CLI in Administration & Architecture

OAUTH Secrets Rotation for Service Principal through Databricks CLI

Rob_Lemmens — Tue, 17 Dec 2024 14:08:21 GMT

I am currently utilizing a specific Service Principal in my DevOps steps to utilize the Databricks CLI. It's using the OAuth tokens with M2M authentication (Authenticate access to Azure Databricks with a service principal using OAuth (OAuth M2M) - Azure Databricks | Microsoft Learn). And I use the Client ID and Client Secret, which I store (as plain text) to the datarbicks config file (.databrickscfg ) for authorization.

I created the secret manually and uploaded it to KeyVault. But I want to programatically renew this Secret every 2 months. Otherwise it is not in line with the security requirements of my company. However, I don't see a straightforward way of doing this through the Databricks CLI. Can someone help with this?

This question is also asked (amongst other questions) in this thread:
Security Consideration for OAUTH Secrets to use Se... - Databricks Community - 78227

Re: OAUTH Secrets Rotation for Service Principal through Databricks CLI

Alberto_Umana — Tue, 17 Dec 2024 14:11:03 GMT

Hi @Rob_Lemmens,

To programmatically renew OAuth secrets for a Service Principal every 2 months, you can follow these steps:

Create a Service Principal and OAuth Secret:

Follow the steps in the Azure Databricks documentation to create a Service Principal and generate an OAuth secret.

Store the Secret in Azure Key Vault:

Store the generated OAuth secret in Azure Key Vault for secure access.

Automate Secret Renewal:

Use Azure Functions or Azure Automation to create a scheduled task that runs every 2 months.
The task should:

Generate a new OAuth secret using the Databricks REST API.
Update the secret in Azure Key Vault.

Update .databrickscfg File:

Ensure your DevOps pipeline retrieves the latest secret from Azure Key Vault and updates the .databrickscfg file before using the Databricks CLI.

Example Azure Function to Renew OAuth Secret

import os

import requests

from azure.identity import DefaultAzureCredential

from azure.keyvault.secrets import SecretClient

# Azure Key Vault details

key_vault_name = os.environ[

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/auth/oauth-m2m

Re: OAUTH Secrets Rotation for Service Principal through Databricks CLI

Rob_Lemmens — Tue, 17 Dec 2024 14:23:26 GMT

Thanks @Alberto_Umana, for the quick reply

So I want to know about step 3. Could you elaborate on that?

Because so far I only see ways to create actual OAuth tokens in Databricks REST API. I don't see how to actually renew the OAuth secret.

Re: OAUTH Secrets Rotation for Service Principal through Databricks CLI

Alberto_Umana — Tue, 17 Dec 2024 14:49:05 GMT

Hi @Rob_Lemmens,

There is no direct method to renew an OAuth secret. Instead, you can create a new OAuth secret and replace the old one. You might need to create an Azure Function or Azure Automation runbook that will execute the renewal process every 2 months, but to replace the token, unfortunately cannot be renewed.

Re: OAUTH Secrets Rotation for Service Principal through Databricks CLI

Rob_Lemmens — Tue, 17 Dec 2024 15:15:29 GMT

Hi @Alberto_Umana,

That's perfectly fine, to replace it. For me replacing the old secret with a new secret is effectively the same as renewing the secret. So could you help me with how to replace the secret?

And if it is simply using the Databricks REST API. I imagine this should also be executable for agents in Azure Devops Pipeline instead of Azure Functions right?

Re: OAUTH Secrets Rotation for Service Principal through Databricks CLI

Rob_Lemmens — Thu, 19 Dec 2024 07:21:30 GMT

This question is not answered yet. Could someone help me with it? Or is it not possible to programatically update oauth secrets through the Databricks REST API?

Re: OAUTH Secrets Rotation for Service Principal through Databricks CLI

Alberto_Umana — Thu, 19 Dec 2024 14:48:59 GMT

Hi @Rob_Lemmens - please refer to the API documentation:

https://docs.databricks.com/api/workspace/secrets/createscope

https://docs.databricks.com/api/workspace/secrets

curl -X POST https://<databricks-instance>/api/2.0/secrets/scopes/create \
-H "Authorization: Bearer <your-access-token>" \
-d '{
"scope": "my-scope",
"initial_manage_principal": "users"
}'

Replace <databricks-instance> with your Databricks workspace URL and <your-access-token> with your Databricks access token.

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/ci-cd/auth-with-azure-devops

Re: OAUTH Secrets Rotation for Service Principal through Databricks CLI

Rob_Lemmens — Fri, 20 Dec 2024 10:34:46 GMT

Hi @Alberto_Umana,

Thanks for reaching out again.

The second point with the link for auth-with-azure-devops seems promising. I will look into it and let you know if it helped!

The first links to the API for the secrets are not the secrets I have been referring to in this thread. I was referring to the OAuth Secret of a Service Principal (I added a screenshot to the comment)

Re: OAUTH Secrets Rotation for Service Principal through Databricks CLI

Alberto_Umana — Mon, 23 Dec 2024 16:56:46 GMT

Hi @Rob_Lemmens,

Thanks for clarifying about the secrets. So this one might help you: https://learn.microsoft.com/en-us/azure/databricks/dev-tools/auth/oauth-m2m

Re: OAUTH Secrets Rotation for Service Principal through Databricks CLI

Rob_Lemmens — Wed, 29 Jan 2025 08:49:10 GMT

After filing a Microsoft Support Ticket through my client they provided me with the solution to the inquiry. There seems to be a undocumented API call that you can do to create this SP Oauth Client Secret and it works perfectly:

curl -X POST --header "Authorization: Bearer <token>" https://<workspace-host-url>/api/2.0/accounts/servicePrincipals/<service-principal-id>/credentials/secrets

This is one of the open issues at Databricks (https://github.com/databricks/terraform-provider-databricks/issues/3363#issuecomment-2122177117)