- 2711 Views
- 7 replies
- 1 kudos
How to install (mssql) drivers to jobcompute?
Hello, I'm having this issue with job-computes:The snippet of the code is as follows: 84 if self.conf["persist_to_sql"]: 85 # persist to sql 86 df_parsed.write.format( 87 "com.microsoft.sqlserver.jdbc.spark" 88...
- 2711 Views
- 7 replies
- 1 kudos
- 1 kudos
For a job compute, you would have to go init script route. Can you please highlight, the cause of the failure of library installation via init script?
- 1 kudos
- 2435 Views
- 1 replies
- 0 kudos
Static IP for existing workspace
Is there a way to have static IP addresses for Azure Databricks without creating new workspace?We have worked a lot in 2 workspaces (dev and main), but now we need static IP addresses for both to work with some APIs. Do we really have to recreate the...
- 2435 Views
- 1 replies
- 0 kudos
- 0 kudos
I don't think so, at least not on Azure. What you need to do depends on how you set up your workspaces. In Azure, if you just use a default install, a NAT gateway is created and configured for you, so you probably already have a static IP.If you us...
- 0 kudos
- 10192 Views
- 11 replies
- 0 kudos
What is the best practice for connecting Power BI to Azure Databricks?
I refer this document to connect Power BI Desktop and Power BI Service to Azure Databricks.Connect Power BI to Azure Databricks - Azure Databricks | Microsoft LearnHowever, I have a couple of quesitions and concerns. Can anyone kindly help?It seems l...
- 10192 Views
- 11 replies
- 0 kudos
- 0 kudos
To securely connect Power BI to Azure Databricks, avoid using PATs and instead configure a Databricks Service Principal with SQL Warehouse access. Power BI Service does not support Client Credential authentication, so Service Principal authentication...
- 0 kudos
- 837 Views
- 1 replies
- 0 kudos
Gcs databricks community
Hello,I would like to know if it is possible to connect my Databricks community account with a Google cloud storage account via a notebook.I tried to connect it via the json key of my gcs service account but the notebook always gives this error when ...
- 837 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @oricaruso To connect to GCS, you typically need to set the service account JSON key in the cluster’s Spark config, not just in the notebook. However, since the Community Edition has several limitations, like the absence of secret scopes, restrict...
- 0 kudos
- 2359 Views
- 3 replies
- 0 kudos
Multiple feature branches per user using Databricks Asset Bundles
I'm currently helping a team migrating to DABs from dbx and they would like to be able to work on multiple features at the same time.What I was able to do is pass the current branch as a variable in the root_path and various names, so when the bundle...
- 2359 Views
- 3 replies
- 0 kudos
- 0 kudos
@cmathieu can you provide an example inserting the branch name? I'm trying to do the same thing
- 0 kudos
- 1120 Views
- 1 replies
- 0 kudos
Resolved! Removing compute policy permissions using Terraform
By default, the "users" and "admins" groups have CAN_USE permission on the Personal Compute policy.I'm using Terraform and would like to prevent regular users from using this policy to create additional compute clusters.I haven't found a way to do th...
- 1120 Views
- 1 replies
- 0 kudos
- 0 kudos
I learned the Personal Compute policy can be turned off at the account level:https://learn.microsoft.com/en-us/azure/databricks/admin/clusters/personal-compute#manage-policy
- 0 kudos
- 1332 Views
- 4 replies
- 0 kudos
Resolved! Shall we opt for multiple worker nodes in dab workflow template if our codebase is based on pandas.
Hi team, I am working in a databricks asset bundle architecture. Added my codebase repo in a workspace. My question to do we need to opt for multiple worker nodes like num_worker_nodes > 1 or autoscale with range of worker nodes if my codebase has mo...
- 1332 Views
- 4 replies
- 0 kudos
- 1185 Views
- 1 replies
- 0 kudos
Resolved! Azure Databricks Outage - Incident ES-1463382 - Any Official RCA Available?
We experienced service disruptions on May 15th, 2025, related to Incident ID ES-1463382.Could anyone from Databricks share the official Root Cause Analysis (RCA) or point us to the correct contact or channel to obtain it?Thanks in advance!
- 1185 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @angiezz! Please raise a support ticket with the Databricks Support Team. They will be able to provide you with the official documentation regarding this incident.
- 0 kudos
- 419 Views
- 1 replies
- 0 kudos
Databricks Cluster Downsizing time
Hello,It seems that the cluster downsizing at our end is occurring rapidly: sometimes the workers go from 5-3 in mere 2 minutes!Is that normal? Can I do something to increase this downsizing time?- Jahanzeb
- 419 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @jahanzebbehan, unfortunately, I don't believe there is any way to decrease the downsizing time for a cluster, as this mostly happens automatically depending on workload volume.Here are some helpful links on autoscaling:https://community.databrick...
- 0 kudos
- 2257 Views
- 5 replies
- 0 kudos
Resolved! Databricks Serverless Job : sudden random failure
Hi, I've been running a job on Azure Databricks serverless, which just does some batch data processing every 4 hours. This job, deployed with bundles has been running fine for weeks, and all of a sudden, yesterday, it started failing with an error th...
- 2257 Views
- 5 replies
- 0 kudos
- 0 kudos
Hey @thibault , Glad to hear it is working again. I don't see any specific mention of a bug internally that would be related to this, but it is likely that it was due to a change in the underlying runtime for serverless compute. This may be one of th...
- 0 kudos
- 4173 Views
- 2 replies
- 1 kudos
azure databricks automatic user provisioning via terraform
Hi community, Azure databricks recently announced a new user management feature (now in public preview) called automatic-identity-management , which allows Azure databricks to access Azure Entra ID directly and grant users and groups permissions and ...
- 4173 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi , I think The automatic identity management feature provisions Azure Entra ID users and groups directly into Databricks. However, Terraform's databricks_group and databricks_group_member resources are designed for managing groups and memberships w...
- 1 kudos
- 2747 Views
- 1 replies
- 0 kudos
workflow not pickingup correct host value (While working with MLflow model registry URI)
Exception: mlflow.exceptions.MlflowException: An API request to https://canada.cloud.databricks.com/api/2.0/mlflow/model-versions/list-artifacts failed due to a timeout. The error message was: HTTPSConnectionPool(host='canada.cloud.databricks.com', p...
- 2747 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @Dharma25! It looks like this post duplicates one you shared earlier. A response has already been provided in the Original thread. I recommend continuing the discussion there to keep the conversation focused and organized.
- 0 kudos
- 1343 Views
- 4 replies
- 2 kudos
Azure databricks workspace Power BI connector type
In Power BI, there's an "Azure Databricks workspace" connector which, unlike the "Azure Databricks" connector, allows you to connect using a service principal defined in Azure Entra ID (rather than within Databricks).While I can create this connector...
- 1343 Views
- 4 replies
- 2 kudos
- 2 kudos
The "Azure Databricks Workspace" connector in Power BI allows authentication using a service principal from Azure Entra ID, providing more secure and scalable access management compared to the traditional personal token-based "Azure Databricks" conne...
- 2 kudos
- 3052 Views
- 1 replies
- 0 kudos
New default notebook format (IPYNB) causes unintended changes on release
Dear Databricks,We have noticed the following issue since the new default notebook format has been set to IPYNB. When we release our code from (for example) DEV to TST using a release pipeline built in Azure DevOps, we see unintended changes popping ...
- 3052 Views
- 1 replies
- 0 kudos
- 0 kudos
Seems like something went wrong with attaching the screenshot. So here we go.
- 0 kudos
- 7693 Views
- 4 replies
- 2 kudos
Internal error. Attach your notebook to a different compute or restart the current compute. java.lan
Internal error. Attach your notebook to a different compute or restart the current compute.java.lang.RuntimeException: abort: DriverClient destroyed at com.databricks.backend.daemon.driver.DriverClient.$anonfun$poll$3(DriverClient.scala:577) at scala...
- 7693 Views
- 4 replies
- 2 kudos
- 2 kudos
The error is caused due to overlap of connectors or instance, if you see an error as below: And you can see Multiple clusters with same name, which is caused due to running the notebook_1 under a cluster attached to it and re-running a notebook_2 wit...
- 2 kudos
