cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

mksmith521
by • New Contributor II
  • 818 Views
  • 3 replies
  • 2 kudos

Resolved! rich colors affected by recent dark mode update

Hello,I use rich Consoles to print DataFrame modification statistics, and I use dark mode. Up until a few days ago, Databricks handled dark mode by inverting rich's colors. That was fine, I just inverted the colors I wanted before passing them into r...

  • 818 Views
  • 3 replies
  • 2 kudos
Latest Reply
Andreas_DBR
Databricks Employee
  • 2 kudos

Hi @mksmith521,   Thanks for the careful report, and for measuring the color with a picker. You're right: bright_red inside a class="nodarkmode" wrapper does not come back as FF0000. What you're seeing is a limitation of the CSS filter capabilities ...

  • 2 kudos
2 More Replies
Andrew_Barlow
by • New Contributor II
  • 391 Views
  • 2 replies
  • 2 kudos

Resolved! AIM configuration on Databricks (AWS)

Hello,We want to manage our users and groups (not service principals) via AIM on our databricks platform which sits on AWS. This is an existing platform with manually created users and groupsIn the documentation it refers to the externalID of both th...

  • 391 Views
  • 2 replies
  • 2 kudos
Latest Reply
Andrew_Barlow
New Contributor II
  • 2 kudos

Thanks Nina - this was really helpful.

  • 2 kudos
1 More Replies
dans-vander
by • New Contributor II
  • 690 Views
  • 2 replies
  • 0 kudos

[FGAC_ON_DEDICATED_COMPUTE_FAILED] Fine-grained access control (FGAC) on dedicated compute failed

Hi everyone, Starting this thread to check if anyone else has seen a similar issue, and sharing the solution because in our case, it was a detail we hadn't noticed it in the documentation. https://docs.databricks.com/aws/en/compute/single-user-fgac P...

dansvander_0-1789651121627.png dansvander_1-1789652090146.png dansvander_2-1789652099168.png
Administration & Architecture
Databricks Administration
serverless
  • 690 Views
  • 2 replies
  • 0 kudos
Latest Reply
saurabh18cs
Honored Contributor III
  • 0 kudos

Hi @dans-vander  By default Databricks Serverless generic compute cannot reach any of private Azure resources. To enable connectivity a private endpoint is needed. Unfortunately setup does not have unlimited private endpoints that can be used with th...

  • 0 kudos
1 More Replies
tedlo042034
by • New Contributor III
  • 711 Views
  • 3 replies
  • 0 kudos

Resolved! "Organization permissions" in DAB's

Hi all,I've created a Databricks app in git, and it is deployed through DAB's. I want my team to have access to it and manage it and view insights and logs thus i have added in the dab CAN_MANAGE permission to my App DAB to the appropriate group.When...

  • 711 Views
  • 3 replies
  • 0 kudos
Latest Reply
tedlo042034
New Contributor III
  • 0 kudos

@Isaac_18 Thanks for this, that makes sense. Now I believe my confusion stems from navigating the documentation. What's the difference between azure docs and aws docs? We are hosting databricks on azure, so have been navigating the azure docs, and ig...

  • 0 kudos
2 More Replies
margarita_shir
by • New Contributor III
  • 1738 Views
  • 5 replies
  • 1 kudos

Private Route53 hosted zone for cloud.databricks.com

We have Tailscale deployed on the same VPC as our Databricks workspaces. To enable PrivateLink for some workspaces, we created a private Route53 hosted zone for cloud.databricks.com with CNAME records pointing workspace hostnames to the PrivateLink e...

  • 1738 Views
  • 5 replies
  • 1 kudos
Latest Reply
suranga
Databricks Partner
  • 1 kudos

Hi @margarita_shir,I don't think this follow-up got an answer, so jumping back in. Yes, keep the Tailscale split DNS on cloud.databricks.com pointing at your VPC resolver.Here's why switching it to privatelink.cloud.databricks.com breaks things. Tail...

  • 1 kudos
4 More Replies
AjayKumarM
by • New Contributor II
  • 714 Views
  • 3 replies
  • 0 kudos

Resolved! Dashboard id is missing in token claim

When I pass multiple filters using "external_value" while embedding the Databricks dashboard, I encounter an error stating that the dashboard ID is missing. For example, passing a single filter works fine, but when I pass multiple filter values (such...

  • 714 Views
  • 3 replies
  • 0 kudos
Latest Reply
Isaac_18
Contributor II
  • 0 kudos

Hi @AjayKumarM, adding to what @data_pulse found. Since /tokeninfo accepts the JSON fine, I'd look at the next call, where you swap the response for the scoped token. That's usually where a JSON-shaped value goes wrong.In the documented flow, step 3 ...

  • 0 kudos
2 More Replies
koosha
by • New Contributor III
  • 504 Views
  • 3 replies
  • 0 kudos

Asset Bundles: how to link a bundle-managed Genie space to an AI/BI dashboard across environments (o

Hi all,I'm managing an AI/BI dashboard and a Genie space with Databricks Asset Bundles (CLI v1.20, engine: direct), deployed to three workspaces (dev / test / prod) via Azure DevOps.The problem When I link the Genie space to the dashboard in the UI, ...

  • 504 Views
  • 3 replies
  • 0 kudos
Latest Reply
Lu_Wang_ENB_DBX
Databricks Employee
  • 0 kudos

This is a current DAB limitation. CLI 1.20 supports Genie spaces with the direct engine, but the documented dashboard resource has no Genie-space binding field. The dashboard JSON file also isn’t a place where bundle resource references are resolved,...

  • 0 kudos
2 More Replies
harsh120
by • New Contributor III
  • 497 Views
  • 2 replies
  • 0 kudos

Cross-region S3 reads fail with 400 Bad Request on Serverless

ProblemWe read Delta tables from an S3 bucket in af-south-1 (opt-in region) from a Databricks workspace in ap-south-1. Queries work on a classic cluster but fail on Serverless / SQL warehouse with the same data and the same IAM instance profile.Same-...

  • 497 Views
  • 2 replies
  • 0 kudos
Latest Reply
Yougasundar
Contributor
  • 0 kudos

Hi,I read this together with the similar eu-west-1 to af-south-1 thread: https://community.databricks.com/t5/data-engineering/cross-region-s3-reads-suddenly-fail-with-400-bad-request-eu-west/td-p/157229Three points:On serverless, the supported way to...

  • 0 kudos
1 More Replies
VarunDatabricks
by • New Contributor
  • 550 Views
  • 3 replies
  • 0 kudos

AzureDatabricks account console error: User has no account or doesn't belong to the specifiedaccount

Hello Databricks Community,I created an Azure Databricks Premium workspace named adb-netflix using my personal Microsoft account.My workspace is working correctly, and I can run notebooks. The command SELECT CURRENT_METASTORE() returns a metastore id...

  • 550 Views
  • 3 replies
  • 0 kudos
Latest Reply
ethan_parmer
New Contributor III
  • 0 kudos

Personal accounts show up in the tenant as guest users. The fix is a native Entra user who signs in once and becomes account admin. What nobody has said yet: new Azure workspaces get Unity Catalog turned on automatically, and the catalog they can alr...

  • 0 kudos
2 More Replies
Dia75
by • New Contributor II
  • 635 Views
  • 2 replies
  • 1 kudos

Resolved! Is a NAT Gateway required for a VNet-injected authentication workspace ?

Hello Community,We are deploying Azure Databricks in France Central with VNet injection, Secure Cluster Connectivity (SCC) and Private Link.We have a dedicated workspace for the browser_authentication private endpoint, as recommended in the Databrick...

  • 635 Views
  • 2 replies
  • 1 kudos
Latest Reply
Dia75
New Contributor II
  • 1 kudos

Hi @SumeshKashyap ,Thank you for the detailed explanationI've also tested this configuration through a technical spike and confirmed that the dedicated browser_authentication workspace works without a NAT Gateway.Your explanation confirms my findings...

  • 1 kudos
1 More Replies
ASH1243434
by • New Contributor II
  • 561 Views
  • 3 replies
  • 1 kudos

Best pattern for ingesting large (multi-GB) zip files from an external SFTP into Databricks

What I'm trying to doPull one zip file per day from an external vendor's SFTP server into Databricks. Most are a few hundred MB, but some are several GB. The zips contain .txt files. Ultimately I just need the zip landed somewhere Databricks can read...

  • 561 Views
  • 3 replies
  • 1 kudos
Latest Reply
datarich
Databricks Employee
  • 1 kudos

Hi, your instinct is right. For multi-GB zips, this isn't the right tool. 1. Is SFTP through databricks.connection supported? Yes. The SFTP connector is part of Lakeflow Connect and is built on Auto Loader. Things to know: databricks.connection need...

  • 1 kudos
2 More Replies
vkondepati
by • Databricks Partner
  • 698 Views
  • 3 replies
  • 4 kudos

Resolved! DBeaver Connection to DBX UC Triggering Queries and SQL Pro Warehouse Failure –EC instances Remain

Hi Databricks Community,I'm experiencing an unusual issue when connecting DBeaver to Databricks Unity Catalog (UC) through a SQL Pro Warehouse and would appreciate insights from anyone who has encountered similar behavior.Current SetupClient: DBeaver...

  • 698 Views
  • 3 replies
  • 4 kudos
Latest Reply
datarich
Databricks Employee
  • 4 kudos

Hi Venkat, these are two separate issues. 1. DBeaver background queries Yes, this is expected. On connect, DBeaver reads metadata to fill the navigator, autocomplete and table info. It does this by running SHOW/DESCRIBE and information_schema queries...

  • 4 kudos
2 More Replies
Mertokcu
by • New Contributor
  • 621 Views
  • 4 replies
  • 5 kudos

How to measure a Dataset row count and size in AI/BI Dashboards?

Dear Databricks Community and experts, I'm creating AI/BI Dashboards using Genie LLM's /importBI function and I want the optimize my dataset by reducing my rows to 100K and size to 100MB. However, since I couldn't find a clear explanation in any reso...

  • 621 Views
  • 4 replies
  • 5 kudos
Latest Reply
datarich
Databricks Employee
  • 5 kudos

  Hi, 100K rows and 100MB aren't hard limits. They're the threshold for browser-side filtering: Under both: the dataset loads into the browser, so filters and cross-filtering are instant. Over either: filtering runs on the SQL warehouse, so every i...

  • 5 kudos
3 More Replies
Abarghodrat
by • New Contributor II
  • 561 Views
  • 1 replies
  • 0 kudos

How to get my Database account verified?

Hi,I am currently on a paid plan (Premium), but I can't find how to get my account verified. I need help and support team is referring me here to this forum to get help.Very interesting, how support team can't help, not sure.Anybody here who can help...

  • 561 Views
  • 1 replies
  • 0 kudos
Latest Reply
DB-RKL
Databricks Employee
  • 0 kudos

Hi Reza, Sorry you’ve been passed between support channels. The three-App limit and automatic stopping are likely caused by the account still being treated as Free Edition. Free Edition is limited to three Apps and may stop them automatically. A Prem...

  • 0 kudos
carl_hsg
by • New Contributor III
  • 426 Views
  • 2 replies
  • 0 kudos

Serverless compute charge on managed ingestion pipeline despite 100 DBUs/day not reached

Implementing the managed ingestion pipeline for D365 using Lakeflow Connect. No technical issues but confusion when it comes to the charges attributed since Lakeflow Connect on our premium workspace is eligible for 100 DBUs per day. When reviewing sy...

  • 426 Views
  • 2 replies
  • 0 kudos
Latest Reply
emma_s
Databricks Employee
  • 0 kudos

Hi, I've just done some digging into this and as far as I can tell there shoulnd't be any charges for the Lakeflow Ingestion until you hit the 100dbu limit. The only thing I want to clarify is that the 100DBU limit is a total for all ingestion pipeli...

  • 0 kudos
1 More Replies