cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Learnit
by New Contributor II
  • 2339 Views
  • 0 replies
  • 0 kudos

Databricks deployment and automation tools comparison.

Hello All, As a newcomer to databricks, I am seeking guidance on automation within databricks environments. What are the best best practices for deployment, and how do Terraform, the REST API, and the databricks SDK compare in terms of advantages and...

  • 2339 Views
  • 0 replies
  • 0 kudos
MasJamei
by New Contributor II
  • 1374 Views
  • 1 replies
  • 0 kudos

Notebook Id level of uniqueness

Hi there,We know that notebook ids are unique. https://docs.databricks.com/en/workspace/workspace-details.html but I want to know in what level they're unique. For example, if Notebook Ids are unique within a workspace, or are they universally unique...

  • 1374 Views
  • 1 replies
  • 0 kudos
141402
by New Contributor II
  • 2681 Views
  • 2 replies
  • 1 kudos

Have "viewer's credential" as default setting in Query access control, instead of "owner's credential"

Hi everybody,I am curious if there is a way to raise the security level by default. Specifically if it is possible to configure the default setting to limit the access of the query, rather than granting it open, as right now from my side the default ...

  • 2681 Views
  • 2 replies
  • 1 kudos
Latest Reply
Ralph_RevoData
New Contributor II
  • 1 kudos

I would like this too. Or perhaps the ability to change this programmatically in bulk. The CLI, SDK and Rest Endpoints don't appear to work and will not let me change this.

  • 1 kudos
1 More Replies
ParameshM
by New Contributor
  • 1541 Views
  • 0 replies
  • 0 kudos

Ubuntu 22 ODBC Connectivity Issue with PHP - SQL error: [unixODBC][Driver Manager]Can't open lib

Dear Friends,I'm having trouble connecting to Databricks ODBC from Ubuntu 22. I followed the steps documented here: https://docs.databricks.com/en/integrations/jdbc-odbc-bi.html#odbc-linuxHere is my odbc.ini file: [ODBC Data Sources] Databricks=Datab...

Administration & Architecture
driver
ODBC
ubuntu
unix
  • 1541 Views
  • 0 replies
  • 0 kudos
gilShin
by Contributor
  • 7274 Views
  • 4 replies
  • 1 kudos

Resolved! DBT job stuck when running on databricks

HiI'm trying to run a DBT job on a databricks instance. The query should be run on the same instance.When I run the job, I get to: Opening a new connection, currently in state initIt is stuck in that phase for a long time. I'm using IP access list wh...

  • 7274 Views
  • 4 replies
  • 1 kudos
Latest Reply
gilShin
Contributor
  • 1 kudos

I recreated the databricks (there's no other way to solve that). If it was a production databricks workspace it was a disaster!I have created a VM with static public IP and added this IP to the IP access list. Hopefully it'll become the last resort i...

  • 1 kudos
3 More Replies
GiggleByte
by New Contributor II
  • 3384 Views
  • 3 replies
  • 0 kudos

Databricks Access Bundles - config data needed by notebook

I have this structure - Folder-1 - the root of databricks access directory. "databricks.yaml" file is in this directoryFolder-1 / Folder-2  has notebooks. One of the notebook, "test-notebook" is used for *job* configuration in databricks.yaml file.Fo...

  • 3384 Views
  • 3 replies
  • 0 kudos
Latest Reply
karthik_p
Esteemed Contributor
  • 0 kudos

@GiggleByte @Yes based on demo test that I have done, it is working as you said. JSON converted yaml config for job setting need to be placed under resources, that yaml has job config setting, it looks  similar to rest api json request converted in f...

  • 0 kudos
2 More Replies
ta
by New Contributor II
  • 8392 Views
  • 7 replies
  • 0 kudos

Can't access to manage account console

Hello!I have got problem with logging into manage account console.After clicking the Manage Account button, it takes me to the Workspace selection window and after selecting any of them, my workspace opens again.The first login was done by Acitve Dir...

  • 8392 Views
  • 7 replies
  • 0 kudos
Latest Reply
wellington
New Contributor III
  • 0 kudos

I'm facing the same problem, I´m admin on my workspace. Please someone from the Databricks team could respond what is happening when users can't access Accounts to create catalog ???????  It stay in infinite loops requesting the workspace name.

  • 0 kudos
6 More Replies
bfrank1972
by New Contributor III
  • 3830 Views
  • 3 replies
  • 0 kudos

Resolved! Small files and discrepancy in S3 vs catalog

Hello all,I'm in the process of optimizing my tables and I'm running into a confusing situation. I have a table named "trace_messages_fg_streaming_event". If I navigate to the Databricks catalog, it shows stats:Size: 6.7GB, Files: 464But when I look ...

bfrank1972_0-1697577884569.png
  • 3830 Views
  • 3 replies
  • 0 kudos
Latest Reply
bfrank1972
New Contributor III
  • 0 kudos

Nevermind, I'm dumb, figured it out, high frequency transactions leave many little files that need to be vacuumed. 

  • 0 kudos
2 More Replies
gilShin
by Contributor
  • 4724 Views
  • 5 replies
  • 2 kudos

Cannot access admin group members

Hi,I'm using Azure databricks premium tier and lately, when I'm creating a new workspace and after that trying to browse to the admin members page, I get the following error:Uncaught Error: No QueryClient set, use QueryClientProvider to set oneReload...

  • 4724 Views
  • 5 replies
  • 2 kudos
Latest Reply
KateGrim
New Contributor II
  • 2 kudos

Same issue here - both Chrome and Firefox.

  • 2 kudos
4 More Replies
gilShin
by Contributor
  • 4107 Views
  • 2 replies
  • 1 kudos

Resolved! Azure databricks issue

Hi,I'm using Azure databricks service and having trouble blocking public access to my workspace.The security groups that are attached to the subnets that databrics is using are being ignored.From what I've learned in the last couple of days, the data...

  • 4107 Views
  • 2 replies
  • 1 kudos
Latest Reply
gilShin
Contributor
  • 1 kudos

HiForgot to mention that I was using standard edition when writing this question but after struggling too long with that we've decided to upgrade to premium and use the IP access list feature and that worked!ThanksGil

  • 1 kudos
1 More Replies
viniaperes
by New Contributor II
  • 1387 Views
  • 0 replies
  • 0 kudos

How can I find my Account Executive?

I opened a support ticket to Databricks asking how I could whitelist an email domain to access Databricks Academy free self-paced training. I was told that I should see my account executive. How can I find it? Could anyone help me with this?

  • 1387 Views
  • 0 replies
  • 0 kudos
DX
by New Contributor II
  • 2903 Views
  • 2 replies
  • 0 kudos

Validation error on cluster_name is empty

Hi,When trying to update the Worker Type for a cluster that is specific for a job, I get a validation error saying the cluster name is empty.The name is present in the UI.In json view, the attribute is set to empty string ("")Swapping for a new clust...

  • 2903 Views
  • 2 replies
  • 0 kudos
Latest Reply
karthik_p
Esteemed Contributor
  • 0 kudos

@DX Is that job cluster/all purpose cluster that you are using for Jobs 

  • 0 kudos
1 More Replies
OvZ
by New Contributor III
  • 1909 Views
  • 2 replies
  • 1 kudos

restrict ip access

Hi guru'sI working on Databricks azure. I have a question is it possible to restrict access (to the workspace url) based on ipadress other than  "enableIpAccessLists": "true"  in Databricks self ?   for example NSG/firewall/conditional access .. ?Tha...

  • 1909 Views
  • 2 replies
  • 1 kudos
Latest Reply
karthik_p
Esteemed Contributor
  • 1 kudos

@OvZ if you are not going to provide access to user either resource level from azure they can not access databricks workspace right, usually recommended approach is to use access list that was suggested by @Retired_mod 

  • 1 kudos
1 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels