cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Deepak_Kandpal
by New Contributor III
  • 9910 Views
  • 3 replies
  • 3 kudos

Resolved! Invalid configuration value detected for fs.azure.account.key with com.crealytics:spark-excel

I have setup my Databricks notebook to use Service Principal to access ADLS using below configuration.service_credential = dbutils.secrets.get(scope="<scope>",key="<service-credential-key>")   spark.conf.set("fs.azure.account.auth.type.<storage-accou...

  • 9910 Views
  • 3 replies
  • 3 kudos
Latest Reply
Harsha_Dbrs
New Contributor II
  • 3 kudos

Below is the implementation of same code in scala:spark.sparkContext.hadoopConfiguration.set("fs.azure.account.key.<accountName>.dfs.core.windows.net",<accountKey>)

  • 3 kudos
2 More Replies
fuselessmatt
by Contributor
  • 7886 Views
  • 4 replies
  • 1 kudos

Accidentally removing the service principal that owns the view seems to put the Unity Catalog in an illegal state. Can you fix this?

I renamed our service principal in Terraform, which forces a replacement where the old service principal is removed and a new principal with the same permission is recreated. The Terraform succeeds to apply, but when I try to run dbt that creates tab...

  • 7886 Views
  • 4 replies
  • 1 kudos
Latest Reply
fuselessmatt
Contributor
  • 1 kudos

This is also true for removing groups before unassigning them (removing and unassigning in Terraform)│ Error: cannot update grants: Could not find principal with name <My Group Name>

  • 1 kudos
3 More Replies
michael_mehrten
by New Contributor III
  • 26934 Views
  • 25 replies
  • 14 kudos

Resolved! How to use Databricks Repos with a service principal for CI/CD in Azure DevOps?

Databricks Repos best-practices recommend using the Repos REST API to update a repo via your git provider. The REST API requires authentication, which can be done one of two ways:A user / personal access tokenA service principal access tokenUsing a u...

  • 26934 Views
  • 25 replies
  • 14 kudos
Latest Reply
martindlarsson
New Contributor III
  • 14 kudos

Having the exact same problem. Did you find a solution @michael_mehrten ?In my case Im using a managed identity so the solution some topics suggest on generating an access token from a Entra ID service principal is not applicable.

  • 14 kudos
24 More Replies
Ravikumashi
by Contributor
  • 6256 Views
  • 8 replies
  • 0 kudos

failed to initialise azure-event-hub with azure AAD(service principal)

We have been trying to authenticate azure-event-hub with azure AD(service principal) instead of shared access key(connection string) and read events from azure-event-hub and it is failing to initialise azure-event-hubs. And throwing no such method ex...

Error message full
  • 6256 Views
  • 8 replies
  • 0 kudos
Latest Reply
Ravikumashi
Contributor
  • 0 kudos

@swathi-dataops I have added ServicePrincipalCredentialsAuth and ServicePrincipalAuthBase as a normal classes instead of creating a separate jar for these 2 classes and packaged them as a part of my project jar.And used the below code for configuring...

  • 0 kudos
7 More Replies
pgruetter
by Contributor
  • 14956 Views
  • 7 replies
  • 1 kudos

Resolved! How to use Service Principal to connect PowerBI to Databrick SQL Warehouse

Hi allI'm struggling to connect PowerBI service to a Databricks SQL Warehouse using a service principal. I'm following mostly this guide.I created a new app registration in the AAD and created a client secret for it.Now I'm particularly struggling wi...

  • 14956 Views
  • 7 replies
  • 1 kudos
Latest Reply
pgruetter
Contributor
  • 1 kudos

In the end, once the Service Principal is properly authorized on the Databricks side, I had to create a Personal Access Token for the Service Principal using the Databricks API. On the Power BI service side I then had to use username = 'token' and as...

  • 1 kudos
6 More Replies
pgruetter
by Contributor
  • 6355 Views
  • 7 replies
  • 2 kudos

Run Task as Service Principal with Code in Azure DevOps Repo

Hi allI have a task of type Notebook, source is Git (Azure DevOps). This task runs fine with my user, but if I change the Owner to a service principal, I get the following error:Run result unavailable: run failed with error message Failed to checkout...

  • 6355 Views
  • 7 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@pgruetter​ :To enable a service principal to access a specific Azure DevOps repository, you need to grant it the necessary permissions at both the organization and repository levels.Here are the steps to grant the service principal the necessary per...

  • 2 kudos
6 More Replies
giuseppegrieco
by New Contributor III
  • 15309 Views
  • 5 replies
  • 6 kudos

Workflow service principle owned can't checkout git repository

I am trying to deploy a workflow where the owner is a service principal and I am using git integration (backend with azure devops), when I run the workflow it says that it doesn't have permissions to checkout the repo.run failed with error message F...

  • 15309 Views
  • 5 replies
  • 6 kudos
Latest Reply
Anonymous
Not applicable
  • 6 kudos

Hi @Giuseppe Grieco​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us ...

  • 6 kudos
4 More Replies
AW
by New Contributor III
  • 12047 Views
  • 3 replies
  • 8 kudos

Resolved! Creating a service principal with admin role on account level in Azure Databricks using Terraform

Dear Community,In the GUI I can grant the admin role to a service principal with a simple switch.How can I achive the same in Terraform? Do you have some code examples?

switch
  • 12047 Views
  • 3 replies
  • 8 kudos
Latest Reply
AW
New Contributor III
  • 8 kudos

Dear @Pat Sienkiewicz​ , works perfectly! It would be so easy it the documentation would be better... Rg Adrian

  • 8 kudos
2 More Replies
NOOR_BASHASHAIK
by Contributor
  • 1377 Views
  • 1 replies
  • 1 kudos

Unity Catalog - addition of account groups/users to workspaces

Hi allwe have set-up metastore, and were doing certain activities as part of MVP.we realized in a particular databricks workspace that was enabled with UC, in admin settings > "Add Groups" section, user groups from other platforms/projects which leve...

  • 1377 Views
  • 1 replies
  • 1 kudos
Latest Reply
Ismail1
New Contributor III
  • 1 kudos

Good question, didn't think of it that way, from my understanding UC uses users pushed from the account console and not from workspaces, One way to restrict would be to restrict other workspaces from using said catalog and also control ACLs with the ...

  • 1 kudos
WillHeyer
by New Contributor II
  • 7274 Views
  • 1 replies
  • 2 kudos

Resolved! Best Practices for PowerBI Connectivity w/ Partner Connect. Access Token w/ Service Principal, Databricks Username w/ Service account, or OAuth?

I'm aware all are possible methods but are all equal? Or is the matter trivial? Thank you so much!

  • 7274 Views
  • 1 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@Will Heyer​ :The best method for Power BI connectivity with Partner Connect depends on your specific use case and requirements. Here are some factors to consider for each method:Access Token with Service Principal: This method uses a client ID and s...

  • 2 kudos
Anuragranjeet
by New Contributor II
  • 2474 Views
  • 5 replies
  • 2 kudos

docs.databricks.com

I want to generate a personal access token on a service principal. I generated a service principal in Azure active directory and using Azure devops pipeline, I got it added to databricks workspace- where I am a work space administrator. After adding ...

  • 2474 Views
  • 5 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Anurag Ranjeet​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...

  • 2 kudos
4 More Replies
akashsharma7119
by Contributor
  • 29176 Views
  • 13 replies
  • 8 kudos

Resolved! Not able to generate Access Token for Service Principal using rest API

I am trying to generate a Databricks token for a service principal (SP). I have created the SP in Azure AD and have used the Databricks rest api to add it as an admin.When using the Databricks rest API "/api/2.0/token-management/on-behalf-of/tokens" ...

  • 29176 Views
  • 13 replies
  • 8 kudos
Latest Reply
callumwhite
New Contributor III
  • 8 kudos

Hi all,I believe I found a temporary fix for this -Generate an AAD token for the service principle in Azure. Follow this guide if you don't know how to -https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/service-prin-aad-toke...

  • 8 kudos
12 More Replies
rsamant07
by New Contributor III
  • 5251 Views
  • 11 replies
  • 2 kudos

Resolved! DBT Job Type Authenticating to Azure Devops for git_source

we are trying to execute the databricks jobs for dbt task type but it is failing to autheticate to git. Problem is job is created using service principal but service principal don't seem to have access to the repo. few questions we have:1) can we giv...

  • 5251 Views
  • 11 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Rahul Samant​ I'm sorry you could not find a solution to your problem in the answers provided.Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.I suggest p...

  • 2 kudos
10 More Replies
sensanjoy
by Contributor
  • 6034 Views
  • 7 replies
  • 3 kudos

Authenticate Databricks REST API and access delta tables from external web service.

Hi All,We do have a requirement to access delta tables from external web service(Web UI). Presently we have tested it through jdbc connection and authenticated using PAT:Ex. jdbc:spark://[DATABRICKS_HOST]:443/default;transportMode=http;ssl=1;httpPath...

  • 6034 Views
  • 7 replies
  • 3 kudos
Latest Reply
sensanjoy
Contributor
  • 3 kudos

Hi @Suteja Kanuri​ , could you please help me with above queries.

  • 3 kudos
6 More Replies
david_bernstein
by New Contributor III
  • 2976 Views
  • 6 replies
  • 0 kudos

DLT autoloader credentials not available error in Azure

I'm just getting started with Databricks and DLTs. I've followed all the docs and tutorials I can find on this and believe I have set up everything in Azure correctly: service principal, paths, and spark configs. When I run a simple DLT autoloader pi...

  • 2976 Views
  • 6 replies
  • 0 kudos
Latest Reply
david_bernstein
New Contributor III
  • 0 kudos

Thank you, I will look into this.

  • 0 kudos
5 More Replies
Labels