- 4625 Views
- 3 replies
- 4 kudos
Resolved! Enable the "Billable Usage Download' Accounts API on our Azure
Hi Databricks Support,Could you please confirm whether the Billable Usage Download endpoint can be enabled on Azure Databricks account, similar to how it’s available on AWS/GCP? If yes, what are the steps that should be followed for the same.. else ...
- 4625 Views
- 3 replies
- 4 kudos
- 3726 Views
- 1 replies
- 1 kudos
Mismatch cuda/cudnn version on Databricks Runtime GPU ML version
I have a cluster on Databricks with configuration Databricks Runtime Version16.4 LTS ML Beta (includes Apache Spark 3.5.2, GPU, Scala 2.12), and another cluster with configuration 16.0 ML (includes Apache Spark 3.5.2, GPU, Scala 2.12). According to...
- 3726 Views
- 1 replies
- 1 kudos
- 1 kudos
There could be library related conflicts in 16.0ML that got fixed in 16.4ML. I would always recommend to use the LTS version. Thanks
- 1 kudos
- 450 Views
- 2 replies
- 1 kudos
DataBricks JDBC driver fails when unsupported property is submitted
Hello!The Databricks jdbc driver applies a property that is not supported by the connector as a Spark server-side property for the client session. How can I avoid that ? With some tools I do not have 100% control, eg they may add a custom jdbc connec...
- 450 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Kaitsu , The documentation mentions, If you specify a property that is not supported by the connector, then the connector attempts to apply the property as a Spark server-side property for the client session. Unlike many other JDBC drivers that ...
- 1 kudos
- 4047 Views
- 2 replies
- 2 kudos
Pre-loading docker images to cluster pool instances still requires docker URL at cluster creation
I am trying to pre-load a docker image to a Databricks cluster pool instance.As per this article I used the REST API to create the cluster pool and defined a custom Azure container registry as the source for the docker images.https://learn.microsoft....
- 4047 Views
- 2 replies
- 2 kudos
- 2 kudos
@NadithK Pre-loading Docker images to cluster pool instances is for performance optimization (faster cluster startup), but you still must specify the Docker image in your cluster configuration. The pre-loading doesn't eliminate the requirement to dec...
- 2 kudos
- 1359 Views
- 4 replies
- 3 kudos
Resolved! Databricks Default package repositories
I have added an extra-index-url in the default package repository in databricks which points to a repository in azure artifact. The libraries from it are getting installed on job cluster but is not working on the all purpose cluster. Below is the rel...
- 1359 Views
- 4 replies
- 3 kudos
- 3 kudos
Hello @tariq! Did the suggestions shared above help address your issue? If so, please consider marking one of the responses as the accepted solution. If you found a different approach that worked for you, sharing it with the community would be really...
- 3 kudos
- 1251 Views
- 1 replies
- 3 kudos
Resolved! Databricks Usage Dashboard - Tagging Networking Costs
Problem OverviewOur team has successfully integrated Azure Databricks Usage Dashboards to monitor platform-related costs. This addition has delivered valuable insights into our spending patterns. However, we've encountered a tagging issue that's prov...
- 1251 Views
- 1 replies
- 3 kudos
- 3 kudos
There is no direct way to tag certain Azure networking resources (such as network interfaces, public IPs, or managed disks) so that their costs inherit custom tags like "projectRole" in cost reports, because many core networking resources either do n...
- 3 kudos
- 443 Views
- 1 replies
- 0 kudos
Why MLFlow version blocks library installation?
I have a Databricks Asset Bundle proyect running and when the MLFlow library released the 3.4.0 version it happened this error:When restricting the MLFlow version to < 3.4.0 it works.The libraries got stuck installing and the process finished in erro...
- 443 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @PabloCSD! Which Databricks Runtime version are you using?
- 0 kudos
- 1838 Views
- 3 replies
- 3 kudos
Resolved! Markdown Cells Do Not Render Consistently
When I am creating a notebook in the UI editor on DataBricks, markdown cells do not always render after I run them. They still appear in 'editing mode'. See the screenshot below, it should have rendered a H1.Again, this behavior is not consistent. So...
- 1838 Views
- 3 replies
- 3 kudos
- 672 Views
- 2 replies
- 0 kudos
When setting up unity catalog a storage account was created with security risk
When i set up databricks unity catalog, i think it automatically set up a storage account. I'm getting recommendations from Azure that this storage account has high risk associated with it. The problem is this resource has a deny assignment on preven...
- 672 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @howardgagan ,Each Azure Databricks workspace has an associated Azure storage account in a managed resource group known as the workspace storage account.This storage account includes workspace system data (job output, system settings, and logs), D...
- 0 kudos
- 870 Views
- 2 replies
- 3 kudos
Any sample code snippet that contains end-to-end OIDC flow
Hello,Is there a sample code snippet that depicts end-to-end OIDC flow - imagine, there exists a service principal, interactive user who connect to an sql warehouse, get authenticated, and run some sql queries as part of a python script (jdbc/odbc) f...
- 870 Views
- 2 replies
- 3 kudos
- 3 kudos
Hello @noorbasha534, did the code snippet shared above help resolve your concern? If it did, please consider marking it as the accepted solution. If you found another approach, please share it with the community so others can benefit as well.
- 3 kudos
- 1367 Views
- 4 replies
- 1 kudos
Resolved! Spark Job fails with No plan for OptimizedForeachBatchFastpath
Hi Team,I am trying to run a job on Databricks cluster 14.3LTS which streams data from parquet to custom sink. I am facing below error. The same code used to work a month back, however i am facing this issue recently.org.apache.spark.SparkException: ...
- 1367 Views
- 4 replies
- 1 kudos
- 1 kudos
Disabling the setting worked for me on 16.4 (serverless):spark.databricks.streaming.forEachBatch.optimized.fastPath.enabled=false But you have to wonder if this is something Databricks are aware of and are looking to fix.
- 1 kudos
- 709 Views
- 1 replies
- 1 kudos
Resolved! Establish Cross cloud connectivity between Azure Databricks and AWS s3
Hello.We have the cross cloud configuration set as below:AWS - VPC, Transit Gateways, AWS Direct ConnectOn Premise Data centerAzure - Vnet, Transit Vnet , Express Route.We are trying to create a Databricks storage credential as below. The AWS IAM obj...
- 709 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @swee ,Thank you for your query.If your storage account is private, you would need to establish a route to that storage account so you can read data.This is because if your storage is private, your storage account will block access to the publi...
- 1 kudos
- 4035 Views
- 8 replies
- 2 kudos
Resolved! Reporting serverless costs to azure costs
So, we've just recently applied serverless budget polices to some of our vector searches and apps. At the moment they're all going to azure under one general tag that we created.However, we needed more definition. So i added the serverless budget pol...
- 4035 Views
- 8 replies
- 2 kudos
- 2 kudos
Billing or set up explicit export pipelines. Check whether your serverless budget policy tags are under a different namespace in Azure, as sometimes they show up nested.
- 2 kudos
- 1512 Views
- 3 replies
- 3 kudos
Resolved! AWS-Databricks' workspace attached to a NCC doesn't generate Egress Stable IPs
I am facing an issue when configuring a Databricks workspace on AWS with a Network Connectivity Configuration (NCC).Even after attaching the NCC, the workspace does not generate Egress Stable IPs as expected.In the workspace configuration tab, under ...
- 1512 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @ricelso ,Sorry to hear you are still facing this issue.This behaviour isn't expected - I would suggest you kindly raise this with your Databricks Account Executive, and they can raise a support request to get this investigated further.Please let ...
- 3 kudos
- 7759 Views
- 24 replies
- 5 kudos
Resolved! Unable to enable Serverless Notebooks
Hello there,I have a Databricks Premium Subscription but am not able to enable Serverless Notebooks (as that option does not seem to exist). I have gone through DB documentation and have Unity Catalog Enabled. I even opened a ticket (00591635) but it...
- 7759 Views
- 24 replies
- 5 kudos
- 5 kudos
We have the same exact thing happening. The /config endpoint shows it is enabled, but we can not select it. The account is several month old. @Walter_C could you help out here as well?
- 5 kudos
