cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Priyam1
by New Contributor III
  • 3654 Views
  • 1 replies
  • 0 kudos

Editing the SPN name in databricks

As a workspace administrator, I'm finding that I cannot change the Service Principal Name (SPN) immediately after registering it through its application ID in Databricks. It appears that Databricks has linked the SPN name with its application ID and ...

  • 3654 Views
  • 1 replies
  • 0 kudos
Latest Reply
Ayushi_Suthar
Databricks Employee
  • 0 kudos

Hi @Priyam1 , Hope you are doing well!  You can change the display name of the Service Principal using the account SCIM APIs : https://docs.databricks.com/api/azure/account/accountserviceprincipals/patch Please let me know if this helps and leave a l...

  • 0 kudos
NadithK
by Contributor
  • 6034 Views
  • 3 replies
  • 3 kudos

Creating a private link for DBFS root storage

I am facing an issue with securing root DBFS storage access from Databricks.As I understand, Azure Databricks creates a default blob storage (a.k.a root storage) during the deployment process which is used for storing logs and telemetry. This storage...

  • 6034 Views
  • 3 replies
  • 3 kudos
Latest Reply
PratikK
Databricks Employee
  • 3 kudos

Hi @NadithK ,You need to create the private endpoint in the resource group where the workspace is deployed and not in the workspace-managed resource group. The workspace-managed resource group has the deny assignment which will not allow to create a ...

  • 3 kudos
2 More Replies
PallaviVetal
by New Contributor II
  • 4507 Views
  • 7 replies
  • 0 kudos

Disable harmful commands at databricks cluster/workspace level

Hi,I want to prevent user from executing harmful commands like dbutils.fs.rm at cluster level/workspace level.Can we do this in databricks? Are there any classes/APIs that we can override?

  • 4507 Views
  • 7 replies
  • 0 kudos
Latest Reply
Nikhil_2800
New Contributor II
  • 0 kudos

Hi ,I got the similar kind of problem too.I followed this put chmod -x /bin/rm in init.sh in workspace and add it at cluster your cluster level init script or global init script.this restricts the users to use rm command.Regards,Nikhil Srivastava@Pal...

  • 0 kudos
6 More Replies
Mumrel
by Contributor
  • 2500 Views
  • 3 replies
  • 0 kudos

Service Principal can be deleted but permissions not managed

On Azure I added a service principal X to my databricks workspace. I therefore had the Service Prinicpal Manager role on that service principal X. I accidentally downgraded my rights to Service Principal User and now can's get my Managers role back. ...

Mumrel_0-1708705677371.png
  • 2500 Views
  • 3 replies
  • 0 kudos
Latest Reply
Mumrel
Contributor
  • 0 kudos

I believe so because the described effect was also observed in another databricks workspace, where I did NOT make the accidental change

  • 0 kudos
2 More Replies
fradetjulien
by New Contributor
  • 2976 Views
  • 1 replies
  • 0 kudos

SSO with Azure Active Directory : Authentication failed

I have a Databricks account with the E2 version of the Databricks platform. The unified login is not enabled on the account because it was created before June 21, 2023.I configured SSO authentication for the account and for a given workspace (separat...

Administration & Architecture
active directory
sso
  • 2976 Views
  • 1 replies
  • 0 kudos
Latest Reply
146404
New Contributor II
  • 0 kudos

where you able to resolve this? we too face same error

  • 0 kudos
SirCrayon
by New Contributor
  • 2132 Views
  • 0 replies
  • 0 kudos

Do shared clusters have multiple drivers?

Hi,I know that with single clusters, theres a single driver node and one driver per cluster. With shared clusters, multiple jobs can run concurrently. Does this still run on a single driver container or multiple driver containers run per application?...

  • 2132 Views
  • 0 replies
  • 0 kudos
100804
by New Contributor II
  • 2411 Views
  • 1 replies
  • 0 kudos

Instance Profile Access Controls

I manage instance profiles assigned to specific user groups. For example, instance profile A provides access solely to group A. Currently, any user within group A has the ability to update the permissions of a cluster using instance profile A, which ...

  • 2411 Views
  • 1 replies
  • 0 kudos
Latest Reply
100804
New Contributor II
  • 0 kudos

 Hi @Retired_mod,Thank you for your guidance. I am following the strategies outlined in steps 1 and 2, and I remain concerned about a specific scenario.Consider instance profile A, which is designed to grant access exclusively to group A. If user A, ...

  • 0 kudos
ossinova
by Contributor II
  • 1820 Views
  • 2 replies
  • 0 kudos

Defaulting or overriding the cluster policy list order

I have numerous cluster policies varying that varies in sizes (Job - xsmall, Job - small, Job - medium...). However, when I create a new job and create a new job cluster the default policy selected from the drop down menu is on the bigger size. Is th...

Job cluster.png
  • 1820 Views
  • 2 replies
  • 0 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 0 kudos

I checked on our sorting, and there does not seem to be any logic in it.They are not sorted by ID or description for sure.

  • 0 kudos
1 More Replies
RaulPino
by New Contributor III
  • 7110 Views
  • 3 replies
  • 1 kudos

Resolved! Networking reduction cost for NATGateway and Shared Catalog

Use case and context:We have a databricks workspace in a specific region, reading and writing files from/to the same region.We also read from a Shared Catalog in a different company, a data provider, which is pointing to multi-region s3 buckets.The r...

Administration & Architecture
natgateway
networking
S3
shared catalog
VPC
  • 7110 Views
  • 3 replies
  • 1 kudos
Latest Reply
RaulPino
New Contributor III
  • 1 kudos

Thanks @Retired_mod for all the suggestions.After some days of monitoring NAT cost, I realized that the implementation of the S3 Gateway Endpoint it was actually working, the problem was that I thought that this change would be reflected right away i...

  • 1 kudos
2 More Replies
6502
by New Contributor III
  • 20456 Views
  • 1 replies
  • 0 kudos

Resolved! Error: default auth: cannot configure default credentials, please check...

Hola all, I'm experiencing a quite strange error. The problem is that and happens inside a GITLAB pipeline:$ databricks current-user meError: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/au...

  • 20456 Views
  • 1 replies
  • 0 kudos
Latest Reply
6502
New Contributor III
  • 0 kudos

Hola Kaniz, the problem is not on Databricks CLI but is due to some interactions happening inside the Gitlab pipeline. According to the documentation reported here: Databricks personal access token authentication | Databricks on AWS ( at the bottom o...

  • 0 kudos
Debi-Moha
by New Contributor II
  • 9742 Views
  • 1 replies
  • 2 kudos

External locations being shared across workspaces

Currently, we have 3 Unity Catalog enabled workspaces sharing the same metastore. Now, when we create an external location or storage credential in any of the workspaces, it gets reflected across all workspaces. We are looking for some best practices...

  • 9742 Views
  • 1 replies
  • 2 kudos
Latest Reply
Allia
Databricks Employee
  • 2 kudos

Hi @Debi-Moha  Currently we do not have a mechanism to isolate the external locations and storage credentials based on workspaces, since the metastore is shared across the workspaces. Please check below document for recommendations on securing extern...

  • 2 kudos
Carsten03
by New Contributor III
  • 17189 Views
  • 2 replies
  • 0 kudos

Bitbucket Cloud Repo Integration with Token

Hey,I am using Bitbucket Cloud and I want to connect my repository to Databricks. I am able to connect with my personal app password but what I am looking for is an authentication of a technical user.I need the integration to point to my dbt repo, wh...

  • 17189 Views
  • 2 replies
  • 0 kudos
Latest Reply
Carsten03
New Contributor III
  • 0 kudos

Hi @Retired_mod,thank you for your response! With this link you provided, I was able to authenticate with Bitbucket Cloud. The solution was to use x-token-auth as a username. I have tried with the generated email address before which didn't work. Tha...

  • 0 kudos
1 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels