cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Lakehouse Architecture


Forum Posts

ilarsen
by Contributor
  • 1645 Views
  • 3 replies
  • 0 kudos

When to add Users Groups or SPs from Account to Workspace

Hi community We are using Unity Catalog, SCIM and Identity Federation, so we have users, groups and service principals at Account level.  In what scenarios do users, groups and service principals need explicitly added to a Workspace?

  • 1645 Views
  • 3 replies
  • 0 kudos
Latest Reply
adamjosh
New Contributor II
  • 0 kudos

In the context of Unity Catalog, SCIM (System for Cross-domain Identity Management), and Identity Federation in a Unity-based environment, users, groups, and service principals may need to be explicitly added to a Workspace in various scenarios based...

  • 0 kudos
2 More Replies
bric-a-brac
by New Contributor
  • 2082 Views
  • 1 replies
  • 0 kudos

Approval required when connecting Tableau Desktop to Azure Databricks

My users want to connect Tableau Desktop to Azure Databricks. I have tested this myself as Administrator of the databricks workspace and it works fine. One of my users can connect to Azure databricks fine and run notebooks. However when he connects T...

image-20240105-123145.png
  • 2082 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, Could you please send the whole screenshot? Is it asking to login to azure and asking Azure's approval or Databricks? 

  • 0 kudos
OwenDB
by New Contributor III
  • 6120 Views
  • 4 replies
  • 3 kudos

Spark jobs not starting for saving

I am having issues with a new workspace that has been created and having trouble diagnosing what the issue is.It seems to be related to compute resources, spark and storage.I am an admin and able to create compute resources. Using SQL warehouses I ca...

  • 6120 Views
  • 4 replies
  • 3 kudos
Latest Reply
OwenDB
New Contributor III
  • 3 kudos

ok looks like it might be this for us....You must have a separate pair of host/container subnets for each workspace that you deploy. It is unsupported to share subnets across workspaces or to deploy other Azure resources on the subnets that are used ...

  • 3 kudos
3 More Replies
Redford
by New Contributor
  • 4046 Views
  • 2 replies
  • 0 kudos

Can I configure Notebook Result Downloads with Databricks CLI , API or Terraform provider ?

I'm Databricks Admin and I'm looking for a solution to automate some Security Workspace settings.Those are:Notebook result downloadSQL result downloadNotebook table clipboard featuresI can't find these options in the Databricks terraform provider, Da...

  • 4046 Views
  • 2 replies
  • 0 kudos
Latest Reply
nkraj
Databricks Employee
  • 0 kudos

Hi @Redford, With the Databricks API, you have the capability to toggle the following features: Enable/Disable Features Notebook result download (key name: enableResultsDownloading)Notebook table clipboard features (key name: enableNotebo...

  • 0 kudos
1 More Replies
Kroy
by Contributor
  • 2674 Views
  • 2 replies
  • 1 kudos

Not able to edit the user group entitlement

I am workspace admin , but when editing the entitlement (allow unrestricted cluster creation) of user group, it is not happening, it gives a prompt to confirm the remove, but evening after confirming it does not get removes. after clicking remove , i...

Kroy_0-1704885328747.png Kroy_1-1704885377008.png
  • 2674 Views
  • 2 replies
  • 1 kudos
Latest Reply
Avvar2022
Contributor
  • 1 kudos

is your workspace untiy catalog enabled?

  • 1 kudos
1 More Replies
Wojciech_BUK
by Valued Contributor III
  • 6623 Views
  • 2 replies
  • 1 kudos

Resolved! Unity Catalog - Lakehouse Federation: Permission to read data from foreign catalogs

I have seup connection "SQL-SV-conn" to SQL Server and based on that connection I have created foreign catalog "FC-SQL-SV".I have granted All permission on CATALOG to developers:Use CatalogUse SchemaSelectBut they can not query table (e.g. by running...

Administration & Architecture
Foreign Catalog
Lakehouse Federation
Unity Catalog
  • 6623 Views
  • 2 replies
  • 1 kudos
Latest Reply
Wojciech_BUK
Valued Contributor III
  • 1 kudos

OK, I have found out the answer in below docummentation:https://learn.microsoft.com/en-us/azure/databricks/query-federation/#limitationsSingle-user access mode is only available for users that own the connection.So when I use e.g. Job Cluster that ru...

  • 1 kudos
1 More Replies
sachinw
by New Contributor II
  • 3153 Views
  • 0 replies
  • 0 kudos

Workspace Model Registry with Unity Catalog

if the "default catalog for the workspace" is to Unity Catalog, how can we access a model from the workspace model registry?I have already tried  mlflow.set_tracking_uri("databricks") but still try to find catalog in UC.  

  • 3153 Views
  • 0 replies
  • 0 kudos
Aria
by New Contributor III
  • 3549 Views
  • 1 replies
  • 1 kudos

Resolved! Security and vulnerability for Azure databricks clusters in Data Plane

Microsoft defender is not supported for azure databricks clusters. Can someone point me to a document which describe how the security vulnerabilities are reported and fixed for azure databricks clusters in data plane.

Administration & Architecture
security vulnerability patch dataplane
  • 3549 Views
  • 1 replies
  • 1 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 1 kudos

Hi Aria,can you check this link?Basically Databricks checks for vulnerabilities and does pentesting etc."Databricks will use commercially reasonable efforts to address critical vulnerabilities within 14 days, high severity within 30 days, and medium ...

  • 1 kudos
scott1
by New Contributor II
  • 3054 Views
  • 1 replies
  • 0 kudos

Automatic UC enablement with existing Unity Catalog

Hello,The environment I'm working in has an existing account and unity catalog in our primary region. These have been primarily PoC, and have existed for a year or so.Now we're building out our IaC and getting ready to move towards production. In new...

  • 3054 Views
  • 1 replies
  • 0 kudos
Latest Reply
" src="" />
This widget could not be displayed.
This widget could not be displayed.
This widget could not be displayed.
  • 0 kudos

This widget could not be displayed.
Hello,The environment I'm working in has an existing account and unity catalog in our primary region. These have been primarily PoC, and have existed for a year or so.Now we're building out our IaC and getting ready to move towards production. In new...

This widget could not be displayed.
  • 0 kudos
This widget could not be displayed.
leelee3000
by Databricks Employee
  • 2630 Views
  • 1 replies
  • 0 kudos

Connect to Salesforce

Curious if there's a Databricks connector for Salesforce on AWS? 

  • 2630 Views
  • 1 replies
  • 0 kudos
Latest Reply
Wojciech_BUK
Valued Contributor III
  • 0 kudos

There is no "databricks" connector like the on you have in Unity Fedarating e.g. for Snowflake.You can use partner ecosystem e.g. Fivetran https://www.fivetran.com/connectors/salesforceto integrate Salesforce data to your Lakehouse. You also have spa...

  • 0 kudos
riturralde-p
by New Contributor II
  • 3314 Views
  • 1 replies
  • 1 kudos

How to allocate costs per SQL query?

By using System Tables (systen.billing.usage) I'm able to identity DBU usage per query, but I'm not able to identify who ran each query because is not part of the table. I'm also aware of query history where all the queries and who ran them is listed...

  • 3314 Views
  • 1 replies
  • 1 kudos
Latest Reply
riturralde-p
New Contributor II
  • 1 kudos

thanks @Retired_mod for the reply, however query_id is not part of the system.billing.usage table, so no way to join them by IDs. What my Databricks account team suggested me is to join them by timestamps since both tables contain a column like that....

  • 1 kudos
lgepp11
by New Contributor III
  • 4391 Views
  • 2 replies
  • 0 kudos

NPIP tunnel setup failure during launch

In AWS with this current error when spinning up the SQL warehouse or personal compute.Backend private link is enabled.Error: NPIP tunnel setup failure during launch. Please try again later and contact Databricks if the problem persists. Instance boot...

  • 4391 Views
  • 2 replies
  • 0 kudos
Latest Reply
User16539034020
Databricks Employee
  • 0 kudos

Hello,  Thanks for contacting Databricks Support.  Based on the error message: NPIP_TUNNEL_SETUP_FAILURE. It indicates that bootstrap failed due to network connectivity issues between the data plane and control plane.  Seems like you have already dow...

  • 0 kudos
1 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels