cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

pablogarcia
by New Contributor II
  • 374 Views
  • 3 replies
  • 2 kudos

Use wheels from volumes in serverless

Hi everyone! I’m working with a job running on Databricks serverless, and I’d like to know how we can load a wheel file that we have stored in a volume, and then use that wheel as a package within the job itself. Any guidance or examples would be app...

  • 374 Views
  • 3 replies
  • 2 kudos
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 2 kudos

Hi @pablogarcia ,You need configure serverless environement to achive that. Refere to below documentation:Configure the serverless environment | Databricks on AWSSpecifically to those sections:- Configure the serverless environment | Databricks on AW...

  • 2 kudos
2 More Replies
niveditha_tr
by New Contributor II
  • 277 Views
  • 2 replies
  • 1 kudos

Subscription management - Can’t see subscription / Access issue

Hi,I recently upgraded my Azure account from Free Trial to Pay-As-You-Go.The Azure portal shows only “Azure subscription 1 – Don’t see a subscription? Switch to another directory.” I have only one directory (“Default Directory”).Please re-associate m...

  • 277 Views
  • 2 replies
  • 1 kudos
Latest Reply
nayan_wylde
Esteemed Contributor
  • 1 kudos

@niveditha_tr Can you please share the resolution here and mark as solution.

  • 1 kudos
1 More Replies
Mendi
by New Contributor
  • 3460 Views
  • 1 replies
  • 0 kudos

Azure Databricks with VNET injection and SCC

Hi,Azure databricks with VNET injection and SCC need to communicate with Azure endpoints for following,Metastore, artifact Blob storage, system tables storage, log Blob storage, and Event Hubs endpoint IP addresses.https://learn.microsoft.com/en-us/a...

  • 3460 Views
  • 1 replies
  • 0 kudos
Latest Reply
Louis_Frolio
Databricks Employee
  • 0 kudos

Hey @Mendi ,  Here’s how connectivity works for Azure Databricks with VNet injection and Secure Cluster Connectivity (SCC) for the endpoints you listed.   Key points from the Microsoft Learn reference The page lists, per region, the FQDNs and ports f...

  • 0 kudos
ceceliac
by New Contributor III
  • 4272 Views
  • 1 replies
  • 2 kudos

Salesforce Marketing Cloud integration

What is the best way to get Salesforce Marketing Cloud data into Databricks? Lakeflow / Federation connectors are limited to Salesforce and Salesforce Data Cloud right now. Are there plans to add Salesforce Marketing Cloud?  The only current option w...

  • 4272 Views
  • 1 replies
  • 2 kudos
Latest Reply
Louis_Frolio
Databricks Employee
  • 2 kudos

Hey @ceceliac ,  Thanks for raising this — here’s the current picture and practical paths you can use today.   What Databricks supports today The Lakehouse Federation connector for Salesforce Data Cloud is available and lets you query Data Cloud tabl...

  • 2 kudos
apjeskeaa
by New Contributor II
  • 775 Views
  • 4 replies
  • 1 kudos

Resolved! Can a Databricks Workspace be renamed after creation ?

A Databricks workspace has already been created with all configurations completed. The customer has now requested to change the workspace name. Is it possible to rename an existing Databricks workspace after creation?

  • 775 Views
  • 4 replies
  • 1 kudos
Latest Reply
jeffreyaven
Databricks Employee
  • 1 kudos

Yes, you can safely rename it. The workspace name is largely cosmetic - it won't affect the actual workspace functionality, API endpoints, or integrations since those all rely on the deployment name/URL (which doesn't change). That said, just a heads...

  • 1 kudos
3 More Replies
sfibich1
by New Contributor II
  • 397 Views
  • 3 replies
  • 1 kudos

Resolved! API call to /api/2.0/serving-endpoints/{name}/ai-gateway does not support tokens or principals

From what I understand of reading the documentation the /api/2.0/serving-endpoints/{name}/ai-gateway supports a "tokens" and a "principals" attribute in the JSON payload.Documentation link: Update AI Gateway of a serving endpoint | Serving endpoints ...

sfibich1_0-1761683609470.png
  • 397 Views
  • 3 replies
  • 1 kudos
Latest Reply
jeffreyaven
Databricks Employee
  • 1 kudos

I have dug a bit deeper on this these properties are supported but not as top level request body fields, instead they are available in object element fields under `rate_limits`. The actual payload looks like:: ```{    "guardrails": { /* ... */ },    ...

  • 1 kudos
2 More Replies
PNC
by New Contributor III
  • 337 Views
  • 1 replies
  • 1 kudos

Using Terraform to GRANT SELECT ON ANY FILE securable

I have a use case where service principals will read .csv files from Azure Storage Account and create views from them. This used to work in our legacy environment but we are currently migrating to Unity Catalog and when we tested our existing jobs we...

  • 337 Views
  • 1 replies
  • 1 kudos
Latest Reply
nayan_wylde
Esteemed Contributor
  • 1 kudos

you can try using this code.resource "databricks_grants" "any_file_select_grant" { principal = "your_user_or_group_name" // Replace with the actual user or group privileges { privilege_type = "SELECT" securable_type = "ANY FILE" } }

  • 1 kudos
MMJ
by New Contributor
  • 326 Views
  • 1 replies
  • 1 kudos

Resolved! Delta share not showing in delta shared with me

Hi Everyone,We just start using Databricks, and we were expecting to receive a Delta Share from a third-party provider. They’ve confirmed that the sharing process has been completed on their end. However, the shared data is not appearing on our porta...

  • 326 Views
  • 1 replies
  • 1 kudos
Latest Reply
jeffreyaven
Databricks Employee
  • 1 kudos

You need USE PROVIDER privileges on the recipient workspaces assigned metastore (or you need to be a metastore admin), you will then see the providers delta sharing org name in SHOW PROVIDERS then you can mount their share as a catalog, let me know h...

  • 1 kudos
yvishal519
by Contributor
  • 4338 Views
  • 3 replies
  • 2 kudos

Resolved! Restricting Catalog and External Location Visibility Across Databricks Workspaces

Hi Databricks Community,I need some guidance regarding catalogs and external locations across multiple environments. Here's my situation:I've set up a resource group (dev-rg) and created a Databricks workspace where I successfully created catalogs (b...

  • 4338 Views
  • 3 replies
  • 2 kudos
Latest Reply
eshwari
New Contributor III
  • 2 kudos

I am facing exact similar issue. I don't want to create separate metastore. and I have added environment name as a prefix to all external locations. All the locations are restricted to their workspaces, so functionality wise everything is fine. my co...

  • 2 kudos
2 More Replies
nodeb
by New Contributor III
  • 917 Views
  • 4 replies
  • 5 kudos

Resolved! Azure Databricks Control Plane connectivity issue after migrating to vWAN

Hello everyone,Recently, I received a client request to migrate our Azure Databricks environment from a Hub-and-Spoke architecture to a vWAN Hub architecture with an NVA (Network Virtual Appliance).Here’s a quick overview of the setup:The Databricks ...

  • 917 Views
  • 4 replies
  • 5 kudos
Latest Reply
nayan_wylde
Esteemed Contributor
  • 5 kudos

@nodeb Can you please mark your reply as solution. It will help other users find the resolution fast.

  • 5 kudos
3 More Replies
amrim
by New Contributor II
  • 318 Views
  • 1 replies
  • 2 kudos

Resolved! Academy: Secondary Email purpose

Hello,I'd like to ask about the functionality of the secondary email.When I try to log-in with it, it prompts me to create a new account, and seems disconnected from my existing account.1. How does it help with keeping access to the account in case a...

  • 318 Views
  • 1 replies
  • 2 kudos
Latest Reply
Advika
Databricks Employee
  • 2 kudos

Hello @amrim! If you try logging in with your secondary email, it will prompt you to create a new account. That’s expected behaviour, as the secondary email is meant for recovery purposes. If you lose access to your primary (registration) email, plea...

  • 2 kudos
magicgrin
by New Contributor II
  • 415 Views
  • 3 replies
  • 2 kudos

Unable to access the account console

Hi,I was trying to add in some IP whitelist under https://docs.databricks.com/aws/en/security/network/front-end/ip-access-list-account#gsc.tab=0 and I have locked myself out.  I cannot log into my account console anymore.  I need help.Your administra...

  • 415 Views
  • 3 replies
  • 2 kudos
Latest Reply
magicgrin
New Contributor II
  • 2 kudos

Yes I am account admin and normally can log into https://accounts.cloud.databricks.com via Okta to be able to manage the account.  However because I forgot to whitelist my IP, I cannot log into the account console to do anything to my account.Databri...

  • 2 kudos
2 More Replies
maikel
by New Contributor III
  • 635 Views
  • 2 replies
  • 0 kudos

A way to get databricks data via Rest API on behalf of user

Hello Databricks Community!I am trying to figure out the best way to get the data from databricks via REST API (or Python SDK but not preferable) but to do not lose information about users permissions during authentication. The use case is that serve...

  • 635 Views
  • 2 replies
  • 0 kudos
Latest Reply
nayan_wylde
Esteemed Contributor
  • 0 kudos

The recommended approach is OAuth 2.0 with On-Behalf-Of (OBO) authentication. This allows your server or app to act on behalf of the user, using their identity and permissions to access Databricks resources.How it works:The user authenticates via OAu...

  • 0 kudos
1 More Replies
Seb_G
by New Contributor III
  • 5021 Views
  • 3 replies
  • 3 kudos

Resolved! Unity Catalog Volume mounting broken by cluster environment variables (http proxy)

Hello all,I have a slightly niche issue here, albeit one that others are likely to run into.Using databricks on Azure, my organisation has included extended our WAN into the cloud, so that all compute clusters are granted a private IP address that ca...

  • 5021 Views
  • 3 replies
  • 3 kudos
Latest Reply
Seb_G
New Contributor III
  • 3 kudos

Unfortunately, the only solution I found was to not use the proxy globally. Good luck!

  • 3 kudos
2 More Replies
satishreddy1326
by New Contributor
  • 216 Views
  • 2 replies
  • 1 kudos

Regarding External Location

If I update the ownership or privileges of an external location, will it have any impact on the associated container? Specifically, while users continue to access the container through the external location during frequent updates, could these change...

Administration & Architecture
Databricks
External Location
  • 216 Views
  • 2 replies
  • 1 kudos
Latest Reply
nayan_wylde
Esteemed Contributor
  • 1 kudos

If you transfer ownership of the external location to another principal, the new owner gains full control.Existing privileges may remain intact unless explicitly revoked.However, if the new owner modifies privileges or policies, it could affect user ...

  • 1 kudos
1 More Replies