- 3458 Views
- 1 replies
- 0 kudos
Databricks Managed MLFlow with Different Unity Catalog for Multi-tenant Production Tracing
Is the Databricks Managed MLFlow only trace LLM traffic through Serving Endpoint? Does it support manual tracing in my LLM application with decorator @mlflow.trace ?Also, How can Databricks Managed MLFlow support multi-tenant cases where traces need ...
- 3458 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @jeffreym9 , Managed MLflow Tracing captures traces in notebooks, local IDEs, jobs, and Serving Endpoints—via autologging or manual instrumentation, so it’s not limited to Serving Endpoints-only. Set your tracking URI to databricks and traces are...
- 0 kudos
- 680 Views
- 3 replies
- 4 kudos
Resolved! Writing data from Azure Databrick to Azure SQL Database
Good day,We have a customer who wanted to query an Azure SQL Database from a Databricks notebook. We have configured a connection and a catalog for him and he is able to query the Azure SQL Database. Now he has a new request. He also wants to write d...
- 680 Views
- 3 replies
- 4 kudos
- 4 kudos
You can customize the below code, that makes use of Spark SQL Server access connector, as per your needs: def PersistRemoteSQLTableFromDF( df: DataFrame, databaseName: str, tableName: str, mode: str = "overwrite", schemaName: str ...
- 4 kudos
- 1083 Views
- 4 replies
- 4 kudos
Resolved! A question about Databricks Fine-grained Access Control (FGAC) cost on dedicated compute
Hi All,recently, while testing Fine-grained Access Control (FGAC) on dedicated compute, I came across something that seems a bit unusual, and I’d like to ask if anyone else has seen similar behavior.I created a view with only one record, and had anot...
- 1083 Views
- 4 replies
- 4 kudos
- 4 kudos
Hi @mark_ott Thank you very much for providing such a detailed and insightful explanation.This clearly resolves our confusion as to why an FGAC query that ran for only a few seconds ultimately incurred the DBU consumption shown on the bill, due to th...
- 4 kudos
- 373 Views
- 1 replies
- 0 kudos
Resolved! Issue Using Private CA Certificates for Databricks Serverless Private Git → On-Prem GitLab Connectio
Hi everyone,I’m trying to properly configure Databricks Serverless Private Git to connect to our on-premises GitLab, but I'm running into issues with private CA certificates.Following the latest Databricks recommendations, our connection to GitLab go...
- 373 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @kfadratek , thanks for the detailed context — Let's take a look at what could be causing the SSL verification to fail with a custome CA in Serverless Private Git and discuss some approaches that might resolve it. What’s likely going wrong B...
- 0 kudos
- 183 Views
- 1 replies
- 0 kudos
AiGatewayConfig non backward compatibly issue from 16.3 to 16.4
We're moving form version 16.3 to version 16.4 LTD, and looks like there is a non backward compatibly issue. This is the import that I have in my codefrom databricks.sdk.service.serving import ( # type: ignore # noqa ServedModelInput, # type:...
- 183 Views
- 1 replies
- 0 kudos
- 0 kudos
The error indicates that AiGatewayConfig cannot be imported from databricks.sdk.service.serving after upgrading from version 16.3 to 16.4 LTD, signaling a breaking change or removal in the SDK. Why This Happens With minor version updates, Databricks ...
- 0 kudos
- 4015 Views
- 1 replies
- 0 kudos
Would it be great if the job workflow supports running docker-based tasks
The current workflow function in Databricks gives a series of options such as DLT, Dbt, python scripts, python files, JAR, etc. It would be good to add a docker file to that and simplify the development process a lot, especially on the unit and integ...
- 4015 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @mo_moattar ! Is this still some functionality you're interested in? If so, can you explain a bit more on the use case you're thinking? I'm happy to add this to our feature requests internally, but I know that the Product team will likely request ...
- 0 kudos
- 4755 Views
- 1 replies
- 0 kudos
Resolved! Databricks Community - Cannot See Reply To My Posts
Databricks Community - Cannot See Reply To My Posts.Am I the only one facing this issue or others are also facing the same.
- 4755 Views
- 1 replies
- 0 kudos
- 0 kudos
@clentin , I know this is a response to an older post, but I'm wondering if you ever got this resolved or not? I am able to view the responses to your initial post, so I took the liberty of adding them as screenshots for you. Hope this helps!
- 0 kudos
- 4378 Views
- 2 replies
- 1 kudos
Automating Version Control for Databricks Workflows
I am currently using Databricks Asset Bundles to manage and deploy workflows. While I have successfully automated the version control for notebooks, I am facing challenges with workflows. Specifically, I am looking to automate the process of fetching...
- 4378 Views
- 2 replies
- 1 kudos
- 1 kudos
Automating the reverse synchronization of Databricks workflow (Job) changes made in the Databricks UI back to a GitHub repository is a significant challenge, mainly due to the intentional directionality and guardrails imposed by Databricks Asset Bund...
- 1 kudos
- 4986 Views
- 8 replies
- 2 kudos
Issue with updating email with SCIM Provisioning
Hi all,For our set-up we have configured SCIM provisioning using Entra ID, group assignment on Azure is dealt with by IdentityIQ Sailpoint, and have enabled SSO for Databricks. It has/is working fine apart from one scenario. The original email assign...
- 4986 Views
- 8 replies
- 2 kudos
- 2 kudos
The other option is to raise a ticket with Databricks Accounts team. Our Databricks team worked on the backend and the new email was synced.
- 2 kudos
- 445 Views
- 3 replies
- 0 kudos
Account level Rest API to list workspaces has suddenly stopped working
We use databricks python sdk in one of our Azure databricks workspace to list all the workspaces present in our tenant. The code was working fine since 6-8 months till yesterday and it has started failing suddenly with error :Endpoint not found for /...
- 445 Views
- 3 replies
- 0 kudos
- 0 kudos
My bad, there was an issue with how the accountClient was created. I was able to resolve this. This is still an issue : I also noticed that databricks rest api documentation no longer shows list workspaces as an available API for Azurewhereas it show...
- 0 kudos
- 3510 Views
- 1 replies
- 0 kudos
Customer Managed VPC: Databricks IP Address Ranges
Hello,how often does Databricks change its public ip addresses (the ones that must be whitelisted in a customer managed vpc) and where can I find them?I found this list, but it seems to be incomplete.We moved from a managed vpc to a customer-managed ...
- 3510 Views
- 1 replies
- 0 kudos
- 0 kudos
Greetings @tom_1 , you’re right to cross-check the published list—here’s how the IPs and ports fit together and where to get the authoritative values. Where to find the current Databricks IPs The official source is the Databricks “IP addresses and d...
- 0 kudos
- 3808 Views
- 1 replies
- 0 kudos
Error on github association
Hello,I'm having an error when trying to link a GitHub account to store some scripts. Looks like my profile keeps on eternal loading too. Does anyone know how I can fix that?
- 3808 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @EdsonDEV , Thanks for the screenshot—your Linked accounts page is showing “Error fetching credentials,” which blocks linking GitHub and can make the settings view spin indefinitely. What typically causes this A broken or stale linked Git cr...
- 0 kudos
- 482 Views
- 2 replies
- 3 kudos
Resolved! Azure Databricks Cluster Pricing
Hi, I am trying to workout a rough total pricing of Azure Databricks Cluster using the following assumption. I want to spin a cluster on D13 v2 vms with 9 executors, so in total 1+9 = 10 nodes. I want to use the cluster for 10 hours a day, 30 hours a...
- 482 Views
- 2 replies
- 3 kudos
- 3 kudos
Here is the simple calculation I use based on dollars and assuming the infra is in EUS.Cost ComponentsAzure VM Cost (D13 v2)On-demand price: $0.741/hour per VMMonthly VM cost:10 VMs×300 hours×$0.741=$2,223Yearly VM cost:10×3600×$0.741=$26,6762 2. Dat...
- 3 kudos
- 4306 Views
- 1 replies
- 1 kudos
How to monitor serverless compute usage in real time
Hello, I'm using Databricks Connect to connect a dash app to my Databricks account. My use case is similar to this example: https://github.com/databricks-demos/dbconnect-examples/tree/main/python/PlotlyI've been able to get everything configured and ...
- 4306 Views
- 1 replies
- 1 kudos
- 1 kudos
There is currently no direct, real-time equivalent in the Databricks UI’s “Compute” tab for monitoring serverless (SQL serverless or Data Engineering serverless) compute usage in the same way as classic clusters, where you see live memory, DBU/hr, an...
- 1 kudos
- 1326 Views
- 1 replies
- 1 kudos
Databricks OAUTH(OIDC) with ORY Network
Hi,we are trying to setup OIDC AUTH for Databricks with our Ory Network account. So far we have been using it without any issues with all of our apps and now we wanted to set it up also for Databricks. Unfortunately after many attempts with different...
- 1326 Views
- 1 replies
- 1 kudos
- 1 kudos
To debug OIDC authentication issues (“oidc_generic_token_failure”) with Databricks using Ory Network as your identity provider, there are several steps and data sources you can leverage for deeper insights. Where to Find Detailed Error Information Da...
- 1 kudos
