cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Mani_DevSecops
by New Contributor II
  • 2456 Views
  • 2 replies
  • 0 kudos

Can't able to enable Saml SSO

We are plannig to implement the architecturePingFederate is Front IDP -> Microsoft Entra is Actual IDP where we have our Users -> SSO Saml  --> Databricks(SP)I have done all the config setups everything is working as expected, the request is Finally ...

  • 2456 Views
  • 2 replies
  • 0 kudos
Latest Reply
MoJaMa
Databricks Employee
  • 0 kudos

This may not be solvable here. Do you have Support via Databricks? You'll need to create a Support Ticket (direct if AWS else MSFT ticket if Azure). If not, ping your Account Team for help. Just saying cos it might require the type of debugging that'...

  • 0 kudos
1 More Replies
fkseki
by Contributor
  • 1291 Views
  • 3 replies
  • 0 kudos

Resolved! Get resource permissions using terraform

Is there a way to retrieve resources (cluster, job, volume, catalog and all other objects) permissions?On terraform docs there's a resource databricks_permissions but I didn't find a data source databricks_permissions, grants ou similar.How can I get...

  • 1291 Views
  • 3 replies
  • 0 kudos
Latest Reply
Ashwin_DSA
Databricks Employee
  • 0 kudos

Hi @fkseki, There isn’t a data "databricks_permissions" (or similar) in the Databricks Terraform, only the databricks_permissions resource, and that resource is authoritative for the full ACL of the object. That means Terraform can’t read the current...

  • 0 kudos
2 More Replies
dmensah
by New Contributor II
  • 1539 Views
  • 2 replies
  • 1 kudos

Resolved! Manage MFA

I can't seem to manage my MFA. It just sends me into a loop. Can someone help me look into this please?

  • 1539 Views
  • 2 replies
  • 1 kudos
Latest Reply
dmensah
New Contributor II
  • 1 kudos

Thanks for your help, doing this in incognito worked,

  • 1 kudos
1 More Replies
pauldb
by New Contributor II
  • 1422 Views
  • 2 replies
  • 0 kudos

AI Runtime Public Preview?

How can I get access to AI runtime? I see that it is in 'Public Preview' but I dont see a feature toggle for it on the account Previews page.https://docs.databricks.com/aws/en/machine-learning/ai-runtime/

  • 1422 Views
  • 2 replies
  • 0 kudos
Latest Reply
edonaire
Contributor III
  • 0 kudos

The confusion comes from AI Runtime having two components with different preview states.Single-node tasks are already open and do not require any toggle. You can just open a notebook, click the Connect dropdown, select Serverless GPU, choose your acc...

  • 0 kudos
1 More Replies
APJESK
by Contributor
  • 1174 Views
  • 2 replies
  • 1 kudos

Resolved! Which role is recommended to create and manage Unity Catalog objects—Workspace Admin or Metastore Ad

Which role is recommended to create and manage Unity Catalog objects (catalog, schema, Storage credentials, External Location)—Workspace Admin or Metastore Admin—and why? 

  • 1174 Views
  • 2 replies
  • 1 kudos
Latest Reply
APJESK
Contributor
  • 1 kudos

I am designing the security model for our Databricks platform and need guidance on role selection for managing Unity Catalog. Which role should be used for creating and managing Unity Catalog objects such as Storage Credentials, External Locations, C...

  • 1 kudos
1 More Replies
dermoritz
by Databricks Partner
  • 1327 Views
  • 5 replies
  • 3 kudos

Spark config ignored in job run settings

i am talking about this setting: i tried so far:spark.executor.cores 8andspark.log.level INFOBoth documented here: https://spark.apache.org/docs/latest/configuration.htmlbut neither i see effect nor i see them set if i check sparkui -> environment ta...

Screenshot 2026-03-25 150815.png Screenshot 2026-03-25 151527.png
  • 1327 Views
  • 5 replies
  • 3 kudos
Latest Reply
emma_s
Databricks Employee
  • 3 kudos

Hi, You're not putting in the wrong place, it's just that Databricks doesn't allow certain configs, because they are managed by Databricks for you. For example your core spark config you've shown above won't be recognised as this is set by selected c...

  • 3 kudos
4 More Replies
abhijit007
by Databricks Partner
  • 2419 Views
  • 2 replies
  • 1 kudos

Resolved! Running Browser Based Agentic Applications on Databricks

Hi,We are evaluating whether it is possible to host a browser‑based agentic application on Databricks.Our application performs frontend UI automation using the browser-use Python library and also exposes FastAPI endpoints to drive a UI..Application O...

  • 2419 Views
  • 2 replies
  • 1 kudos
Latest Reply
Lu_Wang_ENB_DBX
Databricks Employee
  • 1 kudos

TLDR: Databricks Apps/serverless won’t support this pattern; classic compute with Databricks Container Services is your only real option on Databricks, and even that has trade‑offs. For serious browser automation, run it off‑platform and integrate wi...

  • 1 kudos
1 More Replies
luiz_felipe
by New Contributor
  • 1612 Views
  • 1 replies
  • 0 kudos

Power BI Service refresh failures caused by intermittent Databricks credential invalidation

Hi everyone,We are experiencing an intermittent issue in Power BI Service where datasets and dataflows connected directly to Databricks start failing refreshes because the credentials appear to be lost or invalidated, even though no manual changes we...

  • 1612 Views
  • 1 replies
  • 0 kudos
Latest Reply
emma_s
Databricks Employee
  • 0 kudos

Hi, This is a known issue with how the Power BI Service handles OAuth refresh tokens with Databricks. Databricks enables Single-Use Refresh Tokens (SURTs) by default for Power BI integrations. If a token refresh fails mid-flight (network hiccup, timi...

  • 0 kudos
NikeshPurohit
by Databricks Partner
  • 4908 Views
  • 1 replies
  • 1 kudos

Unable to connect Claude connector to Databricks MCP server (Genie)

Hello everyone,I am trying to connect Claude’s Custom Connector (Remote MCP) to a Databricks Managed MCP Server using a Genie space, but I keep encountering an authentication connection error. I would appreciate guidance on whether this is a configur...

  • 4908 Views
  • 1 replies
  • 1 kudos
Latest Reply
emma_s
Databricks Employee
  • 1 kudos

Hi, updating from my previous post to give the latest update. As of two weeks ago, there is a public preview for this, it needs to be enabled in your workspace. You also need to have the Claude IP added to Databricks if you have any IP workspace rest...

  • 1 kudos
RikL
by Databricks Partner
  • 1154 Views
  • 2 replies
  • 2 kudos

Resolved! Automatic Identity Management with Nested Groups and API Access

Hi all,I’m exploring Automatic Identity Management for synchronizing nested groups in Azure Databricks. According to the documentation, this feature supports nested groups: Automatic Identity ManagementHowever, the same article notes that groups sync...

  • 1154 Views
  • 2 replies
  • 2 kudos
Latest Reply
emma_s
Databricks Employee
  • 2 kudos

Hi, I've just had a look internally and there is some discussion about making this functionality available but I can't give you a definitive idea of when this might be.  In terms of workarounds the best one I can find is to use Tarracurl to make raw ...

  • 2 kudos
1 More Replies
margarita_shir
by New Contributor III
  • 1048 Views
  • 4 replies
  • 0 kudos

Private Route53 hosted zone for cloud.databricks.com

We have Tailscale deployed on the same VPC as our Databricks workspaces. To enable PrivateLink for some workspaces, we created a private Route53 hosted zone for cloud.databricks.com with CNAME records pointing workspace hostnames to the PrivateLink e...

  • 1048 Views
  • 4 replies
  • 0 kudos
Latest Reply
margarita_shir
New Contributor III
  • 0 kudos

Thank you for your response. I added a new private hosted zone in Amazon Route 53 for privatelink.cloud.databricks.com, and created an A record for us-east-1.privatelink.cloud.databricks.com pointing to the private endpoint IP.I had a question regard...

  • 0 kudos
3 More Replies
lubiarzm1
by Contributor
  • 2916 Views
  • 3 replies
  • 1 kudos

Resolved! [AZURE] Usage of self managable Storage Account instead Default Databricks File Storage

 Hi Team,I’m encountering an issue with my architecture.I’m using VNet Injection and serverless with NCC. Our organization has a firewall controlling both inbound and outbound traffic.I want to restrict all traffic to private networks only, but I ran...

lubiarzm1_0-1771241572642.png
  • 2916 Views
  • 3 replies
  • 1 kudos
Latest Reply
SteveOstrowski
Databricks Employee
  • 1 kudos

Hi TENDK, That is expected behavior and does not necessarily mean something is wrong. Here is what is happening: When you run nslookup from inside a Databricks notebook, the notebook is executing on a cluster that sits inside the Databricks-managed V...

  • 1 kudos
2 More Replies
bdanielatl
by New Contributor II
  • 3268 Views
  • 4 replies
  • 3 kudos

Resolved! Markdown Cells Do Not Render Consistently

When I am creating a notebook in the UI editor on DataBricks, markdown cells do not always render after I run them. They still appear in 'editing mode'. See the screenshot below, it should have rendered a H1.Again, this behavior is not consistent. So...

bdanielatl_0-1736861864741.png
  • 3268 Views
  • 4 replies
  • 3 kudos
Latest Reply
Saf4Databricks
Contributor
  • 3 kudos

Hi @bdanielatl I had the same issue. Google AI's one suggestion of refreshing the browser helped me resolved the issue. Another suggestion (among other suggestions of Google AI) was to clear the cache of the browser.

  • 3 kudos
3 More Replies
ismaelhenzel
by Valued Contributor
  • 1566 Views
  • 2 replies
  • 2 kudos

Resolved! Terraform folder structure and states

Hi everyone, I have a few questions regarding Terraform folder structure and state management for Databricks, and I'd love to get your opinions.For context, our Databricks environment is already deployed and Unity Catalog is configured. The Terraform...

  • 1566 Views
  • 2 replies
  • 2 kudos
Latest Reply
Ashwin_DSA
Databricks Employee
  • 2 kudos

Hi @ismaelhenzel, In terms of your first question, Terraform automatically loads all *.tf files in the same directory, so it’s common practice to organise them by concern. For example,   envs/ prod/ backend.tf # remote state config ...

  • 2 kudos
1 More Replies
wonoh90
by New Contributor
  • 705 Views
  • 1 replies
  • 0 kudos

Resolved! Accidentally Removed myself as account admin

Dear Support Team, I am currently on a 14-day free trial of the Premium Edition and have accidentally removed myself as the account administrator. As I was the only admin, I am now locked out and unable to log in to the account. Could you please help...

  • 705 Views
  • 1 replies
  • 0 kudos
Latest Reply
Ashwin_DSA
Databricks Employee
  • 0 kudos

Hi @wonoh90 Thanks for reaching out and sorry you’re running into this. Since this involves restoring admin access at the account level, it’s not something the community can fix directly. You’ll need to open a support ticket with Databricks so the su...

  • 0 kudos