topic Re: Automation creation of Databricks Top Level Repositories in Data Engineering

Automation creation of Databricks Top Level Repositories

alefred — Thu, 14 Apr 2022 15:42:06 GMT

I'm looking to automate the creation of Top Level repositories in Databricks however isn't possible using cli or API if this repo is private repository(Azure DevOps Repository) because require setup the token in user setting.

databricks repos create \
   --url https://<orgname>:<PAT_token>@dev.azure.com/<orgname>/<projectname>/_git/<reponame>; \
  --provider azureDevOpsServices \
  --path /Repos/Repository/<reponame> \
  --debug

That's a problem because use a git command like git clone with the same structure works without problems.

git clone https://<orgname>:<PAT_token>@dev.azure.com/<orgname>/<projectname>/_git/<reponame>;

Someone with any idea to solve this challenge?

Re: Automation creation of Databricks Top Level Repositories

alefred — Tue, 19 Apr 2022 10:04:03 GMT

Thanks @Kaniz Fatma , I don't have problem creating PAT, the issue it's create the repo in databricks including the PAT.

PD: Take the consideration I don't wanna use Databricks user setting because everything it's deployed by command line.

Regards,

Luis

Re: Automation creation of Databricks Top Level Repositories

Atanu — Sat, 21 May 2022 13:31:52 GMT

We have API availabe for repos https://docs.databricks.com/dev-tools/api/latest/repos.html#operation/get-repos also we are currently supporting SP.

Step 1: As an admin Create a Service Principal

Use this API SCIM API 2.0 (ServicePrincipals) | Databricks on AWS

Step 2: As an admin create a Databricks token on behalf of the Service Principal

Use this API:

Token Management API 2.0 | Databricks on AWS

Step 3: Using the Service Principal Databricks Token call the Git Credentials API

Add the Git token via this API: Git Credentials API 2.0 | Databricks on AWS

Step 4: Invoke Repos APIs as service principal using the SPNs Databricks Token

Re: Automation creation of Databricks Top Level Repositories

jose_gonzalez — Mon, 13 Jun 2022 21:44:36 GMT

Hi @Luis Arias,

Just a friendly follow-up. Did Atanu's response help you to resolved your question? please let us know