- 5730 Views
- 5 replies
- 1 kudos
Resolved! DBT job stuck when running on databricks
HiI'm trying to run a DBT job on a databricks instance. The query should be run on the same instance.When I run the job, I get to: Opening a new connection, currently in state initIt is stuck in that phase for a long time. I'm using IP access list wh...
- 5730 Views
- 5 replies
- 1 kudos
- 1 kudos
I recreated the databricks (there's no other way to solve that). If it was a production databricks workspace it was a disaster!I have created a VM with static public IP and added this IP to the IP access list. Hopefully it'll become the last resort i...
- 1 kudos
- 1214 Views
- 1 replies
- 0 kudos
How can I find my Account Executive?
I opened a support ticket to Databricks asking how I could whitelist an email domain to access Databricks Academy free self-paced training. I was told that I should see my account executive. How can I find it? Could anyone help me with this?
- 1214 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @viniaperes, Thank you for posting your concern on Community! To expedite your request, please list your concerns on our ticketing portal. Our support staff would be able to act faster on the resolution (our standard resolution time is 24-48 hours...
- 0 kudos
- 2735 Views
- 4 replies
- 1 kudos
Databricks Access Bundles - config data needed by notebook
I have this structure - Folder-1 - the root of databricks access directory. "databricks.yaml" file is in this directoryFolder-1 / Folder-2 has notebooks. One of the notebook, "test-notebook" is used for *job* configuration in databricks.yaml file.Fo...
- 2735 Views
- 4 replies
- 1 kudos
- 1 kudos
@GiggleByte @Yes based on demo test that I have done, it is working as you said. JSON converted yaml config for job setting need to be placed under resources, that yaml has job config setting, it looks similar to rest api json request converted in f...
- 1 kudos
- 4793 Views
- 7 replies
- 0 kudos
Can't access to manage account console
Hello!I have got problem with logging into manage account console.After clicking the Manage Account button, it takes me to the Workspace selection window and after selecting any of them, my workspace opens again.The first login was done by Acitve Dir...
- 4793 Views
- 7 replies
- 0 kudos
- 0 kudos
I'm facing the same problem, I´m admin on my workspace. Please someone from the Databricks team could respond what is happening when users can't access Accounts to create catalog ??????? It stay in infinite loops requesting the workspace name.
- 0 kudos
- 3170 Views
- 4 replies
- 0 kudos
Resolved! Small files and discrepancy in S3 vs catalog
Hello all,I'm in the process of optimizing my tables and I'm running into a confusing situation. I have a table named "trace_messages_fg_streaming_event". If I navigate to the Databricks catalog, it shows stats:Size: 6.7GB, Files: 464But when I look ...
- 3170 Views
- 4 replies
- 0 kudos
- 0 kudos
Nevermind, I'm dumb, figured it out, high frequency transactions leave many little files that need to be vacuumed.
- 0 kudos
- 4096 Views
- 7 replies
- 2 kudos
Cannot access admin group members
Hi,I'm using Azure databricks premium tier and lately, when I'm creating a new workspace and after that trying to browse to the admin members page, I get the following error:Uncaught Error: No QueryClient set, use QueryClientProvider to set oneReload...
- 4096 Views
- 7 replies
- 2 kudos
- 3185 Views
- 3 replies
- 1 kudos
Resolved! Azure databricks issue
Hi,I'm using Azure databricks service and having trouble blocking public access to my workspace.The security groups that are attached to the subnets that databrics is using are being ignored.From what I've learned in the last couple of days, the data...
- 3185 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @gilShin, We value your perspective! Please help our community flourish by identifying the most informative and accurate responses. Your participation makes us stronger.
- 1 kudos
- 2234 Views
- 2 replies
- 0 kudos
Validation error on cluster_name is empty
Hi,When trying to update the Worker Type for a cluster that is specific for a job, I get a validation error saying the cluster name is empty.The name is present in the UI.In json view, the attribute is set to empty string ("")Swapping for a new clust...
- 2234 Views
- 2 replies
- 0 kudos
- 0 kudos
@DX Is that job cluster/all purpose cluster that you are using for Jobs
- 0 kudos
- 1591 Views
- 3 replies
- 1 kudos
restrict ip access
Hi guru'sI working on Databricks azure. I have a question is it possible to restrict access (to the workspace url) based on ipadress other than "enableIpAccessLists": "true" in Databricks self ? for example NSG/firewall/conditional access .. ?Tha...
- 1591 Views
- 3 replies
- 1 kudos
- 1 kudos
@OvZ if you are not going to provide access to user either resource level from azure they can not access databricks workspace right, usually recommended approach is to use access list that was suggested by @Kaniz_Fatma
- 1 kudos
- 8843 Views
- 9 replies
- 5 kudos
Resolved! Setting up Databricks with Unity Catalog using a service principal (instead of managed identity)
Hi,We are attempting to set up Databricks with Unity Catalog (metastore) using a service principal (as opposed to the managed identity).Instructions we are using are here: Create a Unity Catalog metastore - Azure Databricks | Microsoft LearnThe chal...
- 8843 Views
- 9 replies
- 5 kudos
- 5 kudos
Thanks to all for the suggestions. Ultimately, we went with the Managed Identity configuration (after all that investigation). Answers very much appreciated. Thank you.
- 5 kudos
- 5666 Views
- 2 replies
- 0 kudos
Resolved! What is best practice for determining how many workspaces I need?
What is the best practice for determining how many workspaces our company needs? What are the most appropriate boundaries now that we have UC?
- 5666 Views
- 2 replies
- 0 kudos
- 0 kudos
@StephanieAlba adding to @Walter_C no of workspaces will depend on design that you are planning1. you can based on business unit2. you can just go by Dev/Qa/Tst/Prd for CICD Practices wise3. if you go based on above approach, UC schema/catalog wise w...
- 0 kudos
- 1478 Views
- 2 replies
- 0 kudos
Tags are not getting added to the nodes
Tags which were added are not getting added to the EC2 instances,
- 1478 Views
- 2 replies
- 0 kudos
- 0 kudos
@SivaPras_50542 adding to @Kaniz_Fatma are you using clusters from pools, then the behavior will be different . tags wont get inherited from clusters for pools, we need to create tags to workspace level/pool level, please check below linkhttps://docs...
- 0 kudos
- 2902 Views
- 3 replies
- 1 kudos
Resolved! How many workspaces should we have?
There is a default limitation for workspace numbers in a Databricks account, which is 50 for an enterprise account: https://docs.databricks.com/en/resources/limits.html What is the best practice if we need more than 50 workspaces? Will we need more a...
- 2902 Views
- 3 replies
- 1 kudos
- 872 Views
- 1 replies
- 1 kudos
Restricting Spark Connect Behind Premium Plan Paywall?
Quoting the databricks-connect docs, "For Databricks Runtime 13.0 and above, Databricks Connect is now built on open-source Spark Connect." What is odd to me is that a requirement for utilizing this open source Spark feature on Databricks, is Unity C...
- 872 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @gandersen-codes , I recommend reaching out directly to Databricks support by filing a support ticket.
- 1 kudos
- 1476 Views
- 1 replies
- 1 kudos
Help Needed: Grants remaining for removed Groups and Service Principal
We have an issue in a Workspace which is managed by Terraform, a change went in to update the Group and Service Principal (SP) names but due to the internal ordering the Groups and SP were removed and replaced before the Grants were updated.If we now...
- 1476 Views
- 1 replies
- 1 kudos
- 1 kudos
We are experiencing a similar issue, except with the storage credential resource. We created some storage credentials using Terraform, but when trying to destroy them using Terraform they were ignored. So we decided to manually delete the storage cre...
- 1 kudos
