- 1805 Views
- 5 replies
- 0 kudos
Cannot list Clusters using Rest API
I am trying to run the following rest API command from:curl -H "Authorization: Bearer <PAT Code>" -X GET "curl -H "Authorization: Bearer <PAT Code>" -X GET "http://<databricks_workspace>.azuredatabricks.net/api/2.0/clusters/list" When I run the comm...
- 1805 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi, I definitely think it is facing network issues. Its just very difficult to identify, when I am able to successfully ping the instance from the server originating the request.It is something jdbc related, just not sure what.it is
- 0 kudos
- 740 Views
- 1 replies
- 0 kudos
JDBC Connect Time out
Anyone know why I would get the JDBC Connect error below:java.sql.SQLException: [Databricks][JDBCDriver](500593) Communication link failure. Failed to connect to server. Reason: com.databricks.client.jdbc42.internal.apache.http.conn.ConnectTimeoutExc...
- 740 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Lawro, That normally happens whenever there is a network issue or firewall blocking the request. Is it failing consistently and have you tested connectivity to your SQL instance using nc -vz command via a notebook?
- 0 kudos
- 2412 Views
- 3 replies
- 3 kudos
Databricks Apps: Issue with ACLs for apps are disabled or not available in this tier
Hello, I've created a dummy app (using the template) and deployed it in an Azure Databricks premium workspace. It is working fine but is only available for those users with access to the Databricks resource.I would like to change the permissions to "...
- 2412 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi, any help? I've settled in the meantime for an Azure Webapp, but it is a pity that I cannot use this just for a configuration step. Any help is welcomed!
- 3 kudos
- 1509 Views
- 2 replies
- 7 kudos
Databricks Unity Catalog Bug - Reset of Network Connectivity Configuration not possible
The following use case is strange regarding the Network Connectivity Configuration (NCC):I create a Workspace (the NCC is empty)I create a NCCI attach the NCC to the WorkspaceI want to remove the NCC from the Workspace -> not possibleTherefore, I can...
- 1509 Views
- 2 replies
- 7 kudos
- 7 kudos
This is the documented behavior in the REST API:https://docs.databricks.com/api/account/workspaces/updateYou cannot remove a network connectivity configuration from the workspace once attached, you can only switch to another one.
- 7 kudos
- 6265 Views
- 9 replies
- 1 kudos
Resolved! OAUTH Secrets Rotation for Service Principal through Databricks CLI
I am currently utilizing a specific Service Principal in my DevOps steps to utilize the Databricks CLI. It's using the OAuth tokens with M2M authentication (Authenticate access to Azure Databricks with a service principal using OAuth (OAuth M2M) - Az...
- 6265 Views
- 9 replies
- 1 kudos
- 1 kudos
After filing a Microsoft Support Ticket through my client they provided me with the solution to the inquiry. There seems to be a undocumented API call that you can do to create this SP Oauth Client Secret and it works perfectly:curl -X POST --header...
- 1 kudos
- 1103 Views
- 1 replies
- 0 kudos
Resolved! AWS Security Hub - The S3 bucket is shared with an external AWS account
Currently We observe a HIGH Risk warning on the Security Hub of AWS Account were we have been deployed a Private Link Databricks. This warning is related to the permissions associated to the root S3 bucket we use, here an example: { "Version": "...
- 1103 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @ambigus9 - Regarding the external AWS account (414351767826). This is actually a Databricks-owned AWS account, not a random external account. It's essential for Databricks' service to function properly. This account is used by Databricks to man...
- 0 kudos
- 2235 Views
- 3 replies
- 2 kudos
Feature request: Ability to delete local branches in git folders
According to the documentation https://learn.microsoft.com/en-us/azure/databricks/repos/git-operations-with-repos "Local branches in Databricks cannot be deleted, so if you must remove them, you must also delete and reclone the repository."Creating a...
- 2235 Views
- 3 replies
- 2 kudos
- 2 kudos
Also, rather than switching between dev branches, you can create another git folder for the other branches. Users can create a git folder for each dev branch they work on. Those git folders can be deleted after the branches are merged
- 2 kudos
- 3641 Views
- 3 replies
- 1 kudos
Resolved! How to check a specific table for it's VACUUM retention period
I'm looking for a way to query for the VACUUM retention period for a specific table. This does not show up with DESSCRIBE DETAIL <table_name>;
- 3641 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @WWoman ,the default retention period is 7 days and as per documentation it is regulated by 'delta.deletedFileRetentionDuration' table property: If there is no delta.deletedFileRetentionDuration table property it means it uses the default, so 7 ...
- 1 kudos
- 4688 Views
- 2 replies
- 1 kudos
Removal of account admin
Hi, I'm having issues with removing account admin (probably the first one, to which databricks account was related to). Under user management, when I hit the delete user button, it prompts:Either missing permissions to delete <user_email> or deleting...
- 4688 Views
- 2 replies
- 1 kudos
- 1 kudos
This error typically occurs when attempting to remove the 'account owner.' The account owner is the user who originally set up the Databricks account. This entitlement is attached to a user so that there is always at least one account admin who can a...
- 1 kudos
- 1554 Views
- 2 replies
- 0 kudos
Resolved! How to optimize Worklfow job startup time
We want to create a workflow pipeline in which we trigger a Databricks workflow job from AWS. However, the startup time of Databricks workflow jobs on job compute is over 10 minutes, which is causing issues.We would like to either avoid this startup ...
- 1554 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Rahul14Gupta, One option is to use serverless job clusters, which can significantly reduce the startup time. Serverless clusters are designed to start quickly and can be a good fit for workloads that require fast initialization. But you can actua...
- 0 kudos
- 4085 Views
- 1 replies
- 0 kudos
Databricks Bundles - Terraform state management
Hello,I had a look at DABs today and it seems they are using Terraform under the hood. The state is stored in Databricks Workspace, in the bundle deployment directory. Is it possible to use just the state management functionality that DABs must have ...
- 4085 Views
- 1 replies
- 0 kudos
- 0 kudos
While it may be possible to use the state management functionality provided by DABs using Terraform, it would require additional effort to synchronize your code and manage state consistently. The choice would depend on the use case and we should keep...
- 0 kudos
- 1504 Views
- 3 replies
- 3 kudos
How to tag databricks workspace to keep track of AWS resources.
How to add tags to a databricks workspace so that tag propagates to all cloud resources created for or by workspace to keep track of their costs.
- 1504 Views
- 3 replies
- 3 kudos
- 3 kudos
Hello!To add tags to a Databricks workspace and ensure they propagate to all cloud resources created for or by the workspace, log in to your Databricks workspace and navigate to the workspace settings or administration console. Use the tag interface ...
- 3 kudos
- 6166 Views
- 11 replies
- 3 kudos
"Azure Container Does Not Exist" when cloning repositories in Azure Databricks
Good Morning, I need some help with the following issue:I created a new Azure Databricks resource using the vnet-injection procedure. (here) I then proceeded to link my Azure Devops account using a personal account token. If I try to clone a reposito...
- 6166 Views
- 11 replies
- 3 kudos
- 3 kudos
Yes, the container is indeed the container in the storage account deployed by the Databricks instance into the managed resource group. This storage account is part of the managed resource group associated with your Azure Databricks workspace. If the ...
- 3 kudos
- 6815 Views
- 1 replies
- 3 kudos
Databricks Cluster Failed to Start - ADD_NODES_FAILED (Solution)
Lately we encountered the issue that the classic compute clusters could not start. With the help of Databricks team to troubleshoot, we found the issue and get it fixed. So I think writing it here could help other people who would encounter the same ...
- 6815 Views
- 1 replies
- 3 kudos
- 3 kudos
Thanks for sharing. We had the same problem. I missed to add private endpoints to the workspace storage account in the managed resource group. I will also add NCC rules in the Databricks account. Then you don't need the subnets in the firewall.
- 3 kudos
- 1482 Views
- 3 replies
- 3 kudos
private endpoint to non-storage azure resource
I'm trying to set up a ncc and private endpoint for a container app environment in azure. However I get the following error:Error occurred when creating private endpoint rule: : BAD_REQUEST: Can not create Private Link Endpoint with name databricks-x...
- 1482 Views
- 3 replies
- 3 kudos
- 3 kudos
All the azure subscriptions have this registered. Could this not be a azure subscription within the databricks tenant?
- 3 kudos
