cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

GeraldBriyolan
by New Contributor II
  • 1146 Views
  • 3 replies
  • 0 kudos

Databricks Federated Token Exchange Returns HTML Login Page Instead of Access Token(GCP →Databricks)

Hi everyone,I’m trying to implement federated authentication (token exchange) from Google Cloud → Databricks without using a client ID / client secret only using a Google-issued service account token. I have also created a federation policy in Databr...

GeraldBriyolan_0-1764050266136.png
Administration & Architecture
Federation Policy
GCP
Token exchange
  • 1146 Views
  • 3 replies
  • 0 kudos
Latest Reply
WiliamRosa
Databricks Partner
  • 0 kudos

You might want to check whether the issue is related to your federation policy configuration.Try reviewing the following documentation to confirm that your policy is correctly set up (issuer, audiences, and other expected claims):https://docs.databri...

  • 0 kudos
2 More Replies
noorbasha534
by Valued Contributor II
  • 3498 Views
  • 3 replies
  • 0 kudos

Azure VM quota for databricks jobs - demand prediction

Hey folks,a quick check -wanted to gather thoughts on how you manage demand for azure VM quota so you don't run into quota limits issues.In our case, we have several data domains (finance, master data, supply chain...) executing their projects in Dat...

  • 3498 Views
  • 3 replies
  • 0 kudos
Latest Reply
mark_ott
Databricks Employee
  • 0 kudos

Yes, Azure Databricks compute policies let you define “quota-like” limits, but only within Databricks, not Azure subscription quotas themselves. You still rely on Azure’s own quota system for vCPU/VM core limits at the subscription level.​ What you c...

  • 0 kudos
2 More Replies
hdelas
by New Contributor
  • 1100 Views
  • 1 replies
  • 0 kudos

Deploying Jobs in Databricks

How can I use the Databricks Python SDK from azure devops to create or update a job and explicitly assign it to a cluster policy (by policy ID or name)? Could you show me an example where the job definition includes a task and a job cluster that refe...

  • 1100 Views
  • 1 replies
  • 0 kudos
Latest Reply
mark_ott
Databricks Employee
  • 0 kudos

  To use the Databricks Python SDK from Azure DevOps to create or update a job and assign it explicitly to a cluster policy, specify the cluster policy by its ID in the job cluster section of your job definition. This ensures the cluster spawned for ...

  • 0 kudos
Victor2
by New Contributor
  • 1222 Views
  • 1 replies
  • 0 kudos

Unable to create connection in Power platform

When i try to create the connection, I get the error message "Connection test failed. Please review your configuration and try again."Here is the response in the network trace:My connection credentials are correct. So, i'm not sure what i am doing wr...

Victor2_1-1764022073345.png Victor2_4-1764022154704.png
  • 1222 Views
  • 1 replies
  • 0 kudos
Latest Reply
mark_ott
Databricks Employee
  • 0 kudos

The error message "Connection test failed. Please review your configuration and try again." when connecting Databricks to Power Platform can occur due to several common issues, even if your credentials are correct. Key Troubleshooting Steps Double-c...

  • 0 kudos
GeraldBriyolan
by New Contributor II
  • 587 Views
  • 1 replies
  • 0 kudos

Need to create an Identity Federation between my Databricks workspace/account and my GCP account

I am trying to authenticate my Databricks account using the Federation for fetching the data. I have created a service account in GCP, and also using Google Auth, I have generated a token, but I don't know how to exchange the token to authenticate Da...

GeraldBriyolan_0-1763968122617.png
Administration & Architecture
Databricks
Federation Policy
GCP
  • 587 Views
  • 1 replies
  • 0 kudos
Latest Reply
stbjelcevic
Databricks Employee
  • 0 kudos

Hi @GeraldBriyolan , You may need to use a Google ID Token to do what you are trying to do: https://docs.databricks.com/gcp/en/dev-tools/auth/authentication-google-id  

  • 0 kudos
old_school
by New Contributor II
  • 1897 Views
  • 3 replies
  • 0 kudos

Cap on OIDC (max 20) Enable workload identity federation for GitHub Actions

Hi Databricks community,I have followed below page and created Github OIDCs but there seems to be a cap on how many OIDC's a Service Principal can create (20 max). Is there any work around for this or some other solution apart from using Client ID an...

  • 1897 Views
  • 3 replies
  • 0 kudos
Latest Reply
stbjelcevic
Databricks Employee
  • 0 kudos

I can't speak for specifically why, but allowing wildcards creates security risks and most identity providers and standards guidance require exact, pre-registered URLs.

  • 0 kudos
2 More Replies
Raman_Unifeye
by Honored Contributor III
  • 1326 Views
  • 5 replies
  • 3 kudos

Prevent Access to AI Functions Execution

As a workspace Admin, I want to prevent unexpected API costs from unrestricted usage of AI Functions (AI_QUERY() etc.), how can we control that only a particular group-users can execute AI Functions ?I understand the function execution cost can be vi...

  • 1326 Views
  • 5 replies
  • 3 kudos
Latest Reply
Raman_Unifeye
Honored Contributor III
  • 3 kudos

ok, so it has to be done at individual end-point and function level 

  • 3 kudos
4 More Replies
kashif_dev
by New Contributor
  • 1929 Views
  • 1 replies
  • 0 kudos

Azure DB Workspace Not Connected to DB Account Unity Catalog & Admin Console Missing (identity=null)

Hi team,I created a brand-new Azure environment and an Azure Databricks workspace, but the workspace appears to be in classic (legacy) mode and is not connected to a Databricks Account, so Unity Catalog cannot be enabled.Below are all the details and...

  • 1929 Views
  • 1 replies
  • 0 kudos
Latest Reply
Coffee77
Honored Contributor III
  • 0 kudos

I think you need a "corporate" account with Azure Global Administrator role to enable/access Databricks account. For instance, in some of my demo workspaces I can't access to UC with my "hotmail" account. I haven't looked deeper into it so far. So, a...

  • 0 kudos
lubiarzm1
by Contributor
  • 3404 Views
  • 7 replies
  • 1 kudos

Resolved! Deployment of private databricks workspace.

I tried to create configuration of Databricks with Vlan injection and I faced few problem during deploymen.1. I tried to deploy my workspace using IaC and terraform. Whole time I face issue with NSG even when I create configuration as follow in this ...

lubiarzm1_0-1763718040279.png
  • 3404 Views
  • 7 replies
  • 1 kudos
Latest Reply
lubiarzm1
Contributor
  • 1 kudos

All issues was resolvedReady to deploy codelocals { default_tags = { terraform = "true" workload = var.app env = var.environment } } resource "azurerm_databricks_access_connector...

  • 1 kudos
6 More Replies
DArcher
by New Contributor
  • 663 Views
  • 1 replies
  • 0 kudos

Printing Notebook Dashboards

Is it possible to print the tables in a notebook dashboard to a PDF?  I have about 10 tables for stratifications in a dashboard that would be great to print all at once into a clean pdf report.

  • 663 Views
  • 1 replies
  • 0 kudos
Latest Reply
Raman_Unifeye
Honored Contributor III
  • 0 kudos

Hi @DArcher - are you using legacy dashboard or modern Lakeview(AI/BI) dashboard?In the legacy one via notebook, there is no such direct way to export. You will have to do custom python script perhaps to write the output in html and then print to PDF...

  • 0 kudos
ashish_modi
by New Contributor II
  • 3948 Views
  • 6 replies
  • 2 kudos

cloud_infra_costs

I was looking at the system catalog and realized that there is an empty table called cloud_infra_costs. Could you tell me what is this for and why it is empty?  

ashish_modi_0-1730131553537.png
  • 3948 Views
  • 6 replies
  • 2 kudos
Latest Reply
Coffee77
Honored Contributor III
  • 2 kudos

You can also take a look at this built-in cost control dashboard explained in the below video or official databricks documentation at https://docs.databricks.com/aws/en/admin/usage/ . Concerning the dashboard, relevant subject for me was you can insp...

  • 2 kudos
5 More Replies
minkun81
by New Contributor
  • 1810 Views
  • 1 replies
  • 0 kudos

Resolved! Account Creation

I attempted to create my account, but I ran into an issue during the process. Here is what I did: - I visited your website and entered my verification code. - When asked about how I will use the database, I selected “For Work, Set up with my cloud” s...

  • 1810 Views
  • 1 replies
  • 0 kudos
Latest Reply
Advika
Community Manager
  • 0 kudos

Hello @minkun81! It looks like you’re stuck in a verification-code loop. Could you try using a different browser, switching to incognito mode, or clearing your cache and cookies before attempting the login again?Also, please make sure you're followin...

  • 0 kudos
kdhina
by New Contributor II
  • 967 Views
  • 2 replies
  • 2 kudos

Resolved! Databricks Systems Tables Link

When will databricks provide an out-of-box solution that automatically links cluster metadata(system.compute), job & task telemetry(system.lakeflow) and mlflow model metrics(system.mlflow) without requiring custom sql join for training and inferences...

  • 967 Views
  • 2 replies
  • 2 kudos
Latest Reply
AbhaySingh
Databricks Employee
  • 2 kudos

There is no public commitment or release forecast for fully automated cross-system monitoring or UI linking system.compute, system.lakeflow, and system.mlflow without custom queries. There is definitely potential for future releases in this area but ...

  • 2 kudos
1 More Replies
Fabrice_MONNIER
by New Contributor III
  • 2166 Views
  • 3 replies
  • 2 kudos

Resolved! How to change the display name for a Service Principal

Hi,I'm trying to change  the displayName for some EntraID managed service principals.I've tried using the scim API with a PATCH request, I get a HTTP 200 response, but the displayName remains the same.I tried the same with Databricks managed service ...

  • 2166 Views
  • 3 replies
  • 2 kudos
Latest Reply
Raman_Unifeye
Honored Contributor III
  • 2 kudos

@Fabrice_MONNIER - If the name isn't changing for pure Databricks SPs, the issue is almost certainly Account-Level vs. Workspace-Level scope. If Service Principal was created at the Account Console level and then added to the Workspace, the Workspace...

  • 2 kudos
2 More Replies
Swastick
by New Contributor III
  • 467 Views
  • 1 replies
  • 0 kudos

Token report page

Hi All,I'm looking for the API for this token report page in databricks admin page so that I can get this on my notebook.There is a API for workspace level not for account level./api/2.0/token-management/tokensCan anyone point me to right API?

Swastick_0-1763627156046.png
  • 467 Views
  • 1 replies
  • 0 kudos
Latest Reply
Raman_Unifeye
Honored Contributor III
  • 0 kudos

This page is an internal aggregation view; there is no single public API endpoint that generates this "Token Report."The Databricks Account Console performs a proprietary backend query that reaches into all your workspaces to build this view. To repl...

  • 0 kudos