- 202 Views
- 5 replies
- 1 kudos
Resolved! Payment receipts of Databricks payments
Hello experts,I am trying to get receipts for the monthly payments done to Databricks. I need them for the financial department of the organization I am working for. The only billing information I get access to is the usage dashboards and the tables ...
- 202 Views
- 5 replies
- 1 kudos
- 769 Views
- 1 replies
- 1 kudos
Troubleshooting Cluster
We had a failure on a previously running fact table load (our biggest one) and it looked like an executor was failing due to a timeout error. As a test we upped the cluster size and changed the spark.executor.heartbeatinterval to 300s and the spark....
- 769 Views
- 1 replies
- 1 kudos
- 1 kudos
The XXKDA error code is a general indicator for task scheduler issues or SPARK_JOB_CANCELLED.
- 1 kudos
- 67 Views
- 1 replies
- 0 kudos
Community Accounts Getting Deleted Automatically
This is the second time in the same week, the accounts are getting vanished without any prior information. All the saved work has no backup I guess and no one would take responsibility because it a community version.Not a good thing to happen twice, ...
- 67 Views
- 1 replies
- 0 kudos
- 0 kudos
I would suggest to follow steps in https://community.databricks.com/t5/support-faqs/databricks-community-sso-august-3rd-2024/ta-p/78459 , if issue persist please contact databricks-community@databricks.com
- 0 kudos
- 68 Views
- 1 replies
- 0 kudos
Databricks SQL Warehouse Hung - Queries Stuck in Queued State & No Alerts Triggered
We have been facing critical challenges with Databricks SQL Warehouse for the last four weeks. We are using Databricks SQL Warehouse injection from IICS, and we have observed the following issues:SQL Warehouse Going into a Hung State – The SQL Wareho...
- 68 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey @sdheepak The first thing you need to identify is the type of SQL Warehouse you are using in Databricks:• Is it Serverless? If so, it is fully managed by Databricks, and you must contact Databricks support because you won’t have access to logs in...
- 0 kudos
- 928 Views
- 21 replies
- 0 kudos
Resolved! Unable to enable Serverless Notebooks
Hello there,I have a Databricks Premium Subscription but am not able to enable Serverless Notebooks (as that option does not seem to exist). I have gone through DB documentation and have Unity Catalog Enabled. I even opened a ticket (00591635) but it...
- 928 Views
- 21 replies
- 0 kudos
- 0 kudos
I'm also experiencing this issue:- I'm using a Premium Databricks subscription deployed on Azure in January 2025.- I've confirmed that Unity Catalogue is enabled in my subscription.- I've accessed my Account Console. Serveless Compute doesn't present...
- 0 kudos
- 96 Views
- 1 replies
- 0 kudos
JVM Heap Memory Graph - more memory used than available
I'm analyzing the memory usage of my Spark application and I see something strange when checking JVM Heap Memory Graph (see screenshot below). Each line on the graph is representing one executor.Why the memory usage sometimes reaches over 10GB, when ...
- 96 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @dbuserng , The memory usage in your Spark application can exceed the spark.executor.memory setting of 7GB for several reasons: • Off-Heap Memory Usage: Spark allows for off-heap memory allocation, which is not managed by the JVM garbage collector...
- 0 kudos
- 352 Views
- 2 replies
- 0 kudos
Configuration of NCC for Serverless to access SQL server running in a Azure VM
Hi Team, I am following this link to configure NCC for a Serverless compute to access a SQL Server running in a Azure VM. https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/This references to adding privat...
- 352 Views
- 2 replies
- 0 kudos
- 0 kudos
also interested in doing this. Have federated queries for Classic Databricks cluster pointing to SQL server, but can't find documentation for Serverless plane connecting to SQL server on a VM
- 0 kudos
- 546 Views
- 5 replies
- 0 kudos
Cannot list Clusters using Rest API
I am trying to run the following rest API command from:curl -H "Authorization: Bearer <PAT Code>" -X GET "curl -H "Authorization: Bearer <PAT Code>" -X GET "http://<databricks_workspace>.azuredatabricks.net/api/2.0/clusters/list" When I run the comm...
- 546 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi, I definitely think it is facing network issues. Its just very difficult to identify, when I am able to successfully ping the instance from the server originating the request.It is something jdbc related, just not sure what.it is
- 0 kudos
- 774 Views
- 9 replies
- 1 kudos
Resolved! OAUTH Secrets Rotation for Service Principal through Databricks CLI
I am currently utilizing a specific Service Principal in my DevOps steps to utilize the Databricks CLI. It's using the OAuth tokens with M2M authentication (Authenticate access to Azure Databricks with a service principal using OAuth (OAuth M2M) - Az...
- 774 Views
- 9 replies
- 1 kudos
- 1 kudos
After filing a Microsoft Support Ticket through my client they provided me with the solution to the inquiry. There seems to be a undocumented API call that you can do to create this SP Oauth Client Secret and it works perfectly:curl -X POST --header...
- 1 kudos
- 365 Views
- 5 replies
- 0 kudos
Resolved! Updating Workspace Cluster
Hello,My organization is experiencing difficulties updating our Google Kubernetes Engine (GKE) cluster.We've reviewed the official GKE documentation for automated cluster updates, but it appears to primarily focus on AWS integrations. We haven't foun...
- 365 Views
- 5 replies
- 0 kudos
- 281 Views
- 2 replies
- 0 kudos
Databricks App in Azure Databricks with private link cluster (no Public IP)
Hello,I've deployed Azure Databricks with a standard Private Link setup (no public IP). Everything works as expected—I can log in via the private/internal network, create clusters, and manage workloads without any issues.When I create a Databricks Ap...
- 281 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello @Behwar, Did you make sure that your internal DNS is configured to map the web application workspace URL to your front-end VPC endpoint. This involves creating an A-record in your internal DNS that maps the workspace URL directly to the front-e...
- 0 kudos
- 158 Views
- 1 replies
- 0 kudos
Databricks Apps not working in postman
I have a question regarding Databricks Apps. I have deployed my databricks Apps, and its working on my laptop, but when I try to open the same url in my mobile its redirecting to databricks signin page, and also its not working through postman as wel...
- 158 Views
- 1 replies
- 0 kudos
- 0 kudos
Is this issue specifically with the Databricks Apps am I right? Are you getting and erro message?
- 0 kudos
- 490 Views
- 4 replies
- 3 kudos
Resolved! Install Python dependency on job cluster from a privately hosted GitLab repository (HTTPS/SSH)
Hello,We intend to deploy a Databricks workflow based on a Python wheel file which needs to run on a job cluster. There is a dependency declared in pyproject.toml which is another Python project living in a private Gitlab repository. We therefore nee...
- 490 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @andre-h ,As a good alternative you can build the python package (wheel or egg) in your gitlab or github workflows and upload the package to dedicated cloud storage bucket. Then followed by you can specify the cloud storage path of your python lib...
- 3 kudos
- 258 Views
- 1 replies
- 1 kudos
Determining spill from system tables
I'm trying to optimize machine selection (D, E, or L types on Azure) for job clusters and all-purpose compute and am struggling to identify where performance is sagging on account of disk spill. Disk spill would suggest that more memory is needed. ...
- 258 Views
- 1 replies
- 1 kudos
- 1 kudos
For historical diagnostics, you might need to consider setting up a custom logging mechanism that captures these metrics over time and stores them in a persistent storage solution, such as a database or a logging service. This way, you can query and ...
- 1 kudos
- 602 Views
- 8 replies
- 3 kudos
PrivateLink Validation Error - When trying to access to Workspace
We have a workspace that had been deployed on AWS customer architecture using Terraform privatelink: https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-private-link-workspaceThe fact is when we disable the Public Acc...
- 602 Views
- 8 replies
- 3 kudos