- 5357 Views
- 3 replies
- 0 kudos
Only absolute paths are currently supported. Paths must begin with '/'
I am facing the above issue when using the Python Databricks SDK. I retreive the job-definition by "client.jobs.get()" and then try to create it on another workspace with"client.jobs.create()"Therefore the job-definition is correct and working fine o...
- 5357 Views
- 3 replies
- 0 kudos
- 0 kudos
You’re hitting a Jobs validation rule that depends on where the notebook is sourced from. With Git-sourced jobs, notebook paths must be relative; with workspace-sourced jobs, paths must be absolute and start with “/”. If a task’s source is treated as...
- 0 kudos
- 2665 Views
- 2 replies
- 1 kudos
Resolved! Databricks On prem version
Hello, Does Databricks offer an on-premises deployment option?If so, does the on-prem version have any restrictions? If not, is there a way around it? Thank you.
- 2665 Views
- 2 replies
- 1 kudos
- 327 Views
- 1 replies
- 0 kudos
mount cifs volume on all purpose compute results in permission denied
I have all networking already set, nslookup resolves NAS server IP and connectivity is enabled from worker nodes to nas server. I am able to mount the same nas drive outside of databricks, I mean standalone linux vm in the same VPC where worker nodes...
- 327 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello, Could you provide with more information about why you want to attach NAS drive to Databricks cluster, please? I am no expert in Storage. As far as I understand, NAS will suffer with IO and Replication Bottlenecks, when attached to Distributed ...
- 0 kudos
- 626 Views
- 2 replies
- 4 kudos
Resolved! Reaching out to Azure Storage with IP from Private VNET pool
Hey All,Is there a way for Databricks to reach out to Azure Storage using private endpoint?We would like no omit enabling access by "all trusted services".All resources are in the same VNET however when Databrics tries to reach out to Storage instead...
- 626 Views
- 2 replies
- 4 kudos
- 2155 Views
- 4 replies
- 2 kudos
Agent outside databricks communication with databricks MCP server
Hello Community!I have a following use case in my project:User -> AI agent -> MCP Server -> Databricks data from unity catalog.- AI agent is not created in the databricks- MCP server is created in the databricks and should expose tools to get data fr...
- 2155 Views
- 4 replies
- 2 kudos
- 2 kudos
Hello @mark_ott thank you very much for this! It gives me a lot of knowledge! I think that since data is stored in databricks we will go with MCP deployed there as well.I have next portion of questions - can you please tell me how to deploy my mcp se...
- 2 kudos
- 961 Views
- 2 replies
- 1 kudos
Task Hanging issue on DBR 15.4
Hello,I am running strucutred streaming pipeline with 5 models loaded using pyfunc.spark_udf. Lately we have been noticing very strange issue of tasks getting hanged and batch is taking very long time finishing its execution.CPU utilization is around...
- 961 Views
- 2 replies
- 1 kudos
- 1 kudos
On DBR 15.4 the DeadlockDetector: TASK_HANGING message usually just means Spark has noticed some very long-running tasks and is checking for deadlocks. With multiple pyfunc.spark_udf models in a streaming query the tasks often appear “stuck” because ...
- 1 kudos
- 3028 Views
- 4 replies
- 3 kudos
Resolved! Asset bundle vs terraform
I would like to understand the differences between Terraform and Asset Bundles, especially since in some cases, they can do the same thing. I’m not talking about provisioning storage, networking, or the Databricks workspace itself—I know that is Terr...
- 3028 Views
- 4 replies
- 3 kudos
- 3 kudos
First, DAB uses terraform in the background. Having said that, my recommendation is to use DAB for whatever component already included and only other tools for IaC not supported yet or non-databricks specific (private VNets, external storages, etc.) ...
- 3 kudos
- 748 Views
- 3 replies
- 0 kudos
Databricks Federated Token Exchange Returns HTML Login Page Instead of Access Token(GCP →Databricks)
Hi everyone,I’m trying to implement federated authentication (token exchange) from Google Cloud → Databricks without using a client ID / client secret only using a Google-issued service account token. I have also created a federation policy in Databr...
- 748 Views
- 3 replies
- 0 kudos
- 0 kudos
You might want to check whether the issue is related to your federation policy configuration.Try reviewing the following documentation to confirm that your policy is correctly set up (issuer, audiences, and other expected claims):https://docs.databri...
- 0 kudos
- 2162 Views
- 3 replies
- 0 kudos
Azure VM quota for databricks jobs - demand prediction
Hey folks,a quick check -wanted to gather thoughts on how you manage demand for azure VM quota so you don't run into quota limits issues.In our case, we have several data domains (finance, master data, supply chain...) executing their projects in Dat...
- 2162 Views
- 3 replies
- 0 kudos
- 0 kudos
Yes, Azure Databricks compute policies let you define “quota-like” limits, but only within Databricks, not Azure subscription quotas themselves. You still rely on Azure’s own quota system for vCPU/VM core limits at the subscription level. What you c...
- 0 kudos
- 767 Views
- 1 replies
- 0 kudos
Deploying Jobs in Databricks
How can I use the Databricks Python SDK from azure devops to create or update a job and explicitly assign it to a cluster policy (by policy ID or name)? Could you show me an example where the job definition includes a task and a job cluster that refe...
- 767 Views
- 1 replies
- 0 kudos
- 0 kudos
To use the Databricks Python SDK from Azure DevOps to create or update a job and assign it explicitly to a cluster policy, specify the cluster policy by its ID in the job cluster section of your job definition. This ensures the cluster spawned for ...
- 0 kudos
- 864 Views
- 1 replies
- 0 kudos
Unable to create connection in Power platform
When i try to create the connection, I get the error message "Connection test failed. Please review your configuration and try again."Here is the response in the network trace:My connection credentials are correct. So, i'm not sure what i am doing wr...
- 864 Views
- 1 replies
- 0 kudos
- 0 kudos
The error message "Connection test failed. Please review your configuration and try again." when connecting Databricks to Power Platform can occur due to several common issues, even if your credentials are correct. Key Troubleshooting Steps Double-c...
- 0 kudos
- 404 Views
- 1 replies
- 0 kudos
Need to create an Identity Federation between my Databricks workspace/account and my GCP account
I am trying to authenticate my Databricks account using the Federation for fetching the data. I have created a service account in GCP, and also using Google Auth, I have generated a token, but I don't know how to exchange the token to authenticate Da...
- 404 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @GeraldBriyolan , You may need to use a Google ID Token to do what you are trying to do: https://docs.databricks.com/gcp/en/dev-tools/auth/authentication-google-id
- 0 kudos
- 1330 Views
- 3 replies
- 0 kudos
Cap on OIDC (max 20) Enable workload identity federation for GitHub Actions
Hi Databricks community,I have followed below page and created Github OIDCs but there seems to be a cap on how many OIDC's a Service Principal can create (20 max). Is there any work around for this or some other solution apart from using Client ID an...
- 1330 Views
- 3 replies
- 0 kudos
- 0 kudos
I can't speak for specifically why, but allowing wildcards creates security risks and most identity providers and standards guidance require exact, pre-registered URLs.
- 0 kudos
- 878 Views
- 5 replies
- 3 kudos
Prevent Access to AI Functions Execution
As a workspace Admin, I want to prevent unexpected API costs from unrestricted usage of AI Functions (AI_QUERY() etc.), how can we control that only a particular group-users can execute AI Functions ?I understand the function execution cost can be vi...
- 878 Views
- 5 replies
- 3 kudos
- 1474 Views
- 1 replies
- 0 kudos
Azure DB Workspace Not Connected to DB Account Unity Catalog & Admin Console Missing (identity=null)
Hi team,I created a brand-new Azure environment and an Azure Databricks workspace, but the workspace appears to be in classic (legacy) mode and is not connected to a Databricks Account, so Unity Catalog cannot be enabled.Below are all the details and...
- 1474 Views
- 1 replies
- 0 kudos
- 0 kudos
I think you need a "corporate" account with Azure Global Administrator role to enable/access Databricks account. For instance, in some of my demo workspaces I can't access to UC with my "hotmail" account. I haven't looked deeper into it so far. So, a...
- 0 kudos
