cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

alesventus
by Contributor
  • 2539 Views
  • 1 replies
  • 0 kudos

Change users status in account console

One of our users has status = Inactive in Admin setting. I tried to set it to active using rest api and the documentation but seems like api is not changing status.I tested api and I can list users so api setting are ok.api request: import requests i...

  • 2539 Views
  • 1 replies
  • 0 kudos
Latest Reply
arpit
Databricks Employee
  • 0 kudos

@alesventus Can you test the same using CLI or may be try to make the status as inactive for a dummy user? Just want to validate if the API is actually working for you or not.

  • 0 kudos
ashdam
by New Contributor III
  • 2464 Views
  • 1 replies
  • 0 kudos

Installing databricks CLI on windows . sh ??¿?

I am following databricks documentaiton for installing the cli on windows and i found that uses sh command which of course doesnt work.My windows in admin mode says cannot find "sh" application. I could install cgwyn and tools like that but since the...

ashdam_0-1698741976775.png ashdam_1-1698742182892.png
  • 2464 Views
  • 1 replies
  • 0 kudos
Latest Reply
arpit
Databricks Employee
  • 0 kudos

Here is the doc which you can refer:AWS, GCP, and Azure 

  • 0 kudos
Kenthos
by New Contributor II
  • 5520 Views
  • 1 replies
  • 1 kudos

Resolved! Guide to Migrating from AWS to Azure

Hello, I would like to know if we have any documents, checklists, or step-by-step instructions for migrating from Databricks on AWS to Azure.

  • 5520 Views
  • 1 replies
  • 1 kudos
Latest Reply
arpit
Databricks Employee
  • 1 kudos

Yes, Databricks Migration Tool is available on Github

  • 1 kudos
cesarv
by New Contributor
  • 1748 Views
  • 1 replies
  • 0 kudos

Error upgrading Databricks workspace to premium

I tried to update a databricks workspace from Standard to Premium in the Azure portal clicking "Upgrade to Premium" but even when it was supposedly "upgraded", I still can't restrict the use of clusters or modify permissions (I'm an administrator on ...

  • 1748 Views
  • 1 replies
  • 0 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 0 kudos

You might need to open a support ticket with Azure to get more details on this issue

  • 0 kudos
anurag_dev
by New Contributor
  • 4117 Views
  • 1 replies
  • 0 kudos

getting alerts in AWS UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.InsideAWS

Hi folks, I have been using IAM instance profile as a credentials provider to Databricks workspace. I am getting this error "Credentials created exclusively for an EC2 instance using instance role <role-arn> have been used from a remote AWS account 4...

  • 4117 Views
  • 1 replies
  • 0 kudos
Latest Reply
Yeshwanth
Databricks Employee
  • 0 kudos

Hi @anurag_dev  Databricks runs a commit service that coordinates writes to Amazon S3 from multiple clusters. This service runs in the Databricks control plane. (441371677306)The S3 commit service helps guarantee consistency of writes across multiple...

  • 0 kudos
bas
by New Contributor III
  • 12786 Views
  • 9 replies
  • 2 kudos

Resolved! Adding a service principal to workspace fails

Trying to add an Azure Service Principal to a workspace fails, whether via the web UI or the databricks cli. With the databricks, the console gives this:databricks service-principals -p ************* create --application-id ********-****-****-****-**...

  • 12786 Views
  • 9 replies
  • 2 kudos
Latest Reply
brickmaker
New Contributor II
  • 2 kudos

Same issue in AWS Databricks.

  • 2 kudos
8 More Replies
sl79
by New Contributor II
  • 3324 Views
  • 1 replies
  • 1 kudos

Unable to start a SQL cluster

We have a GCP databricks hosted deployment. last night our SQL warehouse failed to start and now just sits with this message:"Cluster Start-up Delayed. Please wait while we continue to try and start the cluster. No action is required from you"It didn...

  • 3324 Views
  • 1 replies
  • 1 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 1 kudos

Could you share the driver logs if you have them??

  • 1 kudos
pawelzak
by Databricks Partner
  • 2348 Views
  • 2 replies
  • 0 kudos

Dashboard programmatical update

Hi,The question that I would like to answer is: How can one replicate dashboards across different environments? Ideally I would like to be able to store the definition of a dashboard in my repository and upload it to Databricks with code: this could ...

  • 2348 Views
  • 2 replies
  • 0 kudos
Latest Reply
shan_chandra
Databricks Employee
  • 0 kudos

@pawelzak  - The Dashboards can not be exported using workspace /export API. 

  • 0 kudos
1 More Replies
m997al
by Contributor III
  • 5590 Views
  • 3 replies
  • 0 kudos

Azure Databricks with metastore, cannot create managed table

We have set up Azure Databricks with Unity Catalog (metastore) in an ADLS Gen2 storage account.Used Managed Identity (Databricks Access Connector) for connection from workspace(s) to ADLS Gen2ADLS Gen2 storage account has Storage Blob Data Owner and ...

m997al_0-1697803440437.png
  • 5590 Views
  • 3 replies
  • 0 kudos
Latest Reply
Kris2
Databricks Partner
  • 0 kudos

@m997al Were you able to resolve this issue? I have same issue as you described. I am able to upload into Volumes but not able to create managed tables in ADLS metastore storage account..

  • 0 kudos
2 More Replies
Matt4
by New Contributor III
  • 4261 Views
  • 2 replies
  • 0 kudos

Resolved! System tables in Azure with private link

Hi,Is there any restriction on what is able to be enabled with System tables when using Azure Databricks with a private link connection?  And is there a way around the restriction (that is secure)?In a public network I am able to enable the following...

  • 4261 Views
  • 2 replies
  • 0 kudos
Latest Reply
Matt4
New Contributor III
  • 0 kudos

Turns out it was an issue with Microsoft somewhere, they've now fixed it and all the system schemas have been enabled that I can enable"schemas":[{"schema":"operational_data","state":"UNAVAILABLE"},{"schema":"access","state":"ENABLE_COMPLETED"},{"sch...

  • 0 kudos
1 More Replies
kartikaybagla
by New Contributor II
  • 7852 Views
  • 3 replies
  • 1 kudos

Unable to access all schemas in system catalog

https://docs.databricks.com/en/administration-guide/system-tables/index.htmlBased on this documentation, there should be a bunch of schemas in the system catalog and when trying to list them all, I can only see the INFORMATION_SCHEMA in the system ca...

  • 7852 Views
  • 3 replies
  • 1 kudos
Latest Reply
SanketMehta
Databricks Partner
  • 1 kudos

Hi Team, If you have any particular solution for this, then please share as I am also unable to access any system tables I enabled in 2023 August now!!

  • 1 kudos
2 More Replies
pawelzak
by Databricks Partner
  • 2019 Views
  • 1 replies
  • 0 kudos

Dashboard update through API

Hi,I would like to create / update dashboard definition based on the json file. How can one do it? I tried the following:databricks api post /api/2.0/preview/sql/dashboards/$dashboard_id --json @file.jsonBut it does not update the widgetsdatabricks a...

  • 2019 Views
  • 1 replies
  • 0 kudos
bric-a-brac
by New Contributor
  • 3019 Views
  • 1 replies
  • 0 kudos

Approval required when connecting Tableau Desktop to Azure Databricks

My users want to connect Tableau Desktop to Azure Databricks. I have tested this myself as Administrator of the databricks workspace and it works fine. One of my users can connect to Azure databricks fine and run notebooks. However when he connects T...

image-20240105-123145.png
  • 3019 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, Could you please send the whole screenshot? Is it asking to login to azure and asking Azure's approval or Databricks? 

  • 0 kudos
OwenDB
by New Contributor III
  • 8030 Views
  • 4 replies
  • 3 kudos

Spark jobs not starting for saving

I am having issues with a new workspace that has been created and having trouble diagnosing what the issue is.It seems to be related to compute resources, spark and storage.I am an admin and able to create compute resources. Using SQL warehouses I ca...

  • 8030 Views
  • 4 replies
  • 3 kudos
Latest Reply
OwenDB
New Contributor III
  • 3 kudos

ok looks like it might be this for us....You must have a separate pair of host/container subnets for each workspace that you deploy. It is unsupported to share subnets across workspaces or to deploy other Azure resources on the subnets that are used ...

  • 3 kudos
3 More Replies