cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

sl79
by New Contributor II
  • 2851 Views
  • 1 replies
  • 1 kudos

Unable to start a SQL cluster

We have a GCP databricks hosted deployment. last night our SQL warehouse failed to start and now just sits with this message:"Cluster Start-up Delayed. Please wait while we continue to try and start the cluster. No action is required from you"It didn...

  • 2851 Views
  • 1 replies
  • 1 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 1 kudos

Could you share the driver logs if you have them??

  • 1 kudos
pawelzak
by New Contributor III
  • 1886 Views
  • 2 replies
  • 0 kudos

Dashboard programmatical update

Hi,The question that I would like to answer is: How can one replicate dashboards across different environments? Ideally I would like to be able to store the definition of a dashboard in my repository and upload it to Databricks with code: this could ...

  • 1886 Views
  • 2 replies
  • 0 kudos
Latest Reply
shan_chandra
Databricks Employee
  • 0 kudos

@pawelzak  - The Dashboards can not be exported using workspace /export API. 

  • 0 kudos
1 More Replies
m997al
by Contributor III
  • 4872 Views
  • 3 replies
  • 0 kudos

Azure Databricks with metastore, cannot create managed table

We have set up Azure Databricks with Unity Catalog (metastore) in an ADLS Gen2 storage account.Used Managed Identity (Databricks Access Connector) for connection from workspace(s) to ADLS Gen2ADLS Gen2 storage account has Storage Blob Data Owner and ...

m997al_0-1697803440437.png
  • 4872 Views
  • 3 replies
  • 0 kudos
Latest Reply
Kris2
New Contributor II
  • 0 kudos

@m997al Were you able to resolve this issue? I have same issue as you described. I am able to upload into Volumes but not able to create managed tables in ADLS metastore storage account..

  • 0 kudos
2 More Replies
Matt4
by New Contributor III
  • 3666 Views
  • 2 replies
  • 0 kudos

Resolved! System tables in Azure with private link

Hi,Is there any restriction on what is able to be enabled with System tables when using Azure Databricks with a private link connection?  And is there a way around the restriction (that is secure)?In a public network I am able to enable the following...

  • 3666 Views
  • 2 replies
  • 0 kudos
Latest Reply
Matt4
New Contributor III
  • 0 kudos

Turns out it was an issue with Microsoft somewhere, they've now fixed it and all the system schemas have been enabled that I can enable"schemas":[{"schema":"operational_data","state":"UNAVAILABLE"},{"schema":"access","state":"ENABLE_COMPLETED"},{"sch...

  • 0 kudos
1 More Replies
kartikaybagla
by New Contributor II
  • 6969 Views
  • 3 replies
  • 1 kudos

Unable to access all schemas in system catalog

https://docs.databricks.com/en/administration-guide/system-tables/index.htmlBased on this documentation, there should be a bunch of schemas in the system catalog and when trying to list them all, I can only see the INFORMATION_SCHEMA in the system ca...

  • 6969 Views
  • 3 replies
  • 1 kudos
Latest Reply
SanketMehta
New Contributor II
  • 1 kudos

Hi Team, If you have any particular solution for this, then please share as I am also unable to access any system tables I enabled in 2023 August now!!

  • 1 kudos
2 More Replies
pawelzak
by New Contributor III
  • 1653 Views
  • 1 replies
  • 0 kudos

Dashboard update through API

Hi,I would like to create / update dashboard definition based on the json file. How can one do it? I tried the following:databricks api post /api/2.0/preview/sql/dashboards/$dashboard_id --json @file.jsonBut it does not update the widgetsdatabricks a...

  • 1653 Views
  • 1 replies
  • 0 kudos
bric-a-brac
by New Contributor
  • 2688 Views
  • 1 replies
  • 0 kudos

Approval required when connecting Tableau Desktop to Azure Databricks

My users want to connect Tableau Desktop to Azure Databricks. I have tested this myself as Administrator of the databricks workspace and it works fine. One of my users can connect to Azure databricks fine and run notebooks. However when he connects T...

image-20240105-123145.png
  • 2688 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, Could you please send the whole screenshot? Is it asking to login to azure and asking Azure's approval or Databricks? 

  • 0 kudos
OwenDB
by New Contributor III
  • 7333 Views
  • 4 replies
  • 3 kudos

Spark jobs not starting for saving

I am having issues with a new workspace that has been created and having trouble diagnosing what the issue is.It seems to be related to compute resources, spark and storage.I am an admin and able to create compute resources. Using SQL warehouses I ca...

  • 7333 Views
  • 4 replies
  • 3 kudos
Latest Reply
OwenDB
New Contributor III
  • 3 kudos

ok looks like it might be this for us....You must have a separate pair of host/container subnets for each workspace that you deploy. It is unsupported to share subnets across workspaces or to deploy other Azure resources on the subnets that are used ...

  • 3 kudos
3 More Replies
Kroy
by Contributor
  • 4288 Views
  • 2 replies
  • 1 kudos

Not able to edit the user group entitlement

I am workspace admin , but when editing the entitlement (allow unrestricted cluster creation) of user group, it is not happening, it gives a prompt to confirm the remove, but evening after confirming it does not get removes. after clicking remove , i...

Kroy_0-1704885328747.png Kroy_1-1704885377008.png
  • 4288 Views
  • 2 replies
  • 1 kudos
Latest Reply
Avvar2022
Contributor
  • 1 kudos

is your workspace untiy catalog enabled?

  • 1 kudos
1 More Replies
Wojciech_BUK
by Valued Contributor III
  • 9404 Views
  • 2 replies
  • 1 kudos

Resolved! Unity Catalog - Lakehouse Federation: Permission to read data from foreign catalogs

I have seup connection "SQL-SV-conn" to SQL Server and based on that connection I have created foreign catalog "FC-SQL-SV".I have granted All permission on CATALOG to developers:Use CatalogUse SchemaSelectBut they can not query table (e.g. by running...

Administration & Architecture
Foreign Catalog
Lakehouse Federation
Unity Catalog
  • 9404 Views
  • 2 replies
  • 1 kudos
Latest Reply
Wojciech_BUK
Valued Contributor III
  • 1 kudos

OK, I have found out the answer in below docummentation:https://learn.microsoft.com/en-us/azure/databricks/query-federation/#limitationsSingle-user access mode is only available for users that own the connection.So when I use e.g. Job Cluster that ru...

  • 1 kudos
1 More Replies
sachinw
by New Contributor II
  • 4017 Views
  • 0 replies
  • 1 kudos

Workspace Model Registry with Unity Catalog

if the "default catalog for the workspace" is to Unity Catalog, how can we access a model from the workspace model registry?I have already tried  mlflow.set_tracking_uri("databricks") but still try to find catalog in UC.  

  • 4017 Views
  • 0 replies
  • 1 kudos
Aria
by New Contributor III
  • 6468 Views
  • 1 replies
  • 1 kudos

Resolved! Security and vulnerability for Azure databricks clusters in Data Plane

Microsoft defender is not supported for azure databricks clusters. Can someone point me to a document which describe how the security vulnerabilities are reported and fixed for azure databricks clusters in data plane.

Administration & Architecture
security vulnerability patch dataplane
  • 6468 Views
  • 1 replies
  • 1 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 1 kudos

Hi Aria,can you check this link?Basically Databricks checks for vulnerabilities and does pentesting etc."Databricks will use commercially reasonable efforts to address critical vulnerabilities within 14 days, high severity within 30 days, and medium ...

  • 1 kudos