- 2265 Views
- 2 replies
- 0 kudos
Validation error on cluster_name is empty
Hi,When trying to update the Worker Type for a cluster that is specific for a job, I get a validation error saying the cluster name is empty.The name is present in the UI.In json view, the attribute is set to empty string ("")Swapping for a new clust...
- 2265 Views
- 2 replies
- 0 kudos
- 0 kudos
@DX Is that job cluster/all purpose cluster that you are using for Jobs
- 0 kudos
- 1609 Views
- 3 replies
- 1 kudos
restrict ip access
Hi guru'sI working on Databricks azure. I have a question is it possible to restrict access (to the workspace url) based on ipadress other than "enableIpAccessLists": "true" in Databricks self ? for example NSG/firewall/conditional access .. ?Tha...
- 1609 Views
- 3 replies
- 1 kudos
- 1 kudos
@OvZ if you are not going to provide access to user either resource level from azure they can not access databricks workspace right, usually recommended approach is to use access list that was suggested by @Kaniz_Fatma
- 1 kudos
- 8923 Views
- 9 replies
- 5 kudos
Resolved! Setting up Databricks with Unity Catalog using a service principal (instead of managed identity)
Hi,We are attempting to set up Databricks with Unity Catalog (metastore) using a service principal (as opposed to the managed identity).Instructions we are using are here: Create a Unity Catalog metastore - Azure Databricks | Microsoft LearnThe chal...
- 8923 Views
- 9 replies
- 5 kudos
- 5 kudos
Thanks to all for the suggestions. Ultimately, we went with the Managed Identity configuration (after all that investigation). Answers very much appreciated. Thank you.
- 5 kudos
- 5706 Views
- 2 replies
- 0 kudos
Resolved! What is best practice for determining how many workspaces I need?
What is the best practice for determining how many workspaces our company needs? What are the most appropriate boundaries now that we have UC?
- 5706 Views
- 2 replies
- 0 kudos
- 0 kudos
@StephanieAlba adding to @Walter_C no of workspaces will depend on design that you are planning1. you can based on business unit2. you can just go by Dev/Qa/Tst/Prd for CICD Practices wise3. if you go based on above approach, UC schema/catalog wise w...
- 0 kudos
- 1501 Views
- 2 replies
- 0 kudos
Tags are not getting added to the nodes
Tags which were added are not getting added to the EC2 instances,
- 1501 Views
- 2 replies
- 0 kudos
- 0 kudos
@SivaPras_50542 adding to @Kaniz_Fatma are you using clusters from pools, then the behavior will be different . tags wont get inherited from clusters for pools, we need to create tags to workspace level/pool level, please check below linkhttps://docs...
- 0 kudos
- 2959 Views
- 3 replies
- 1 kudos
Resolved! How many workspaces should we have?
There is a default limitation for workspace numbers in a Databricks account, which is 50 for an enterprise account: https://docs.databricks.com/en/resources/limits.html What is the best practice if we need more than 50 workspaces? Will we need more a...
- 2959 Views
- 3 replies
- 1 kudos
- 879 Views
- 1 replies
- 1 kudos
Restricting Spark Connect Behind Premium Plan Paywall?
Quoting the databricks-connect docs, "For Databricks Runtime 13.0 and above, Databricks Connect is now built on open-source Spark Connect." What is odd to me is that a requirement for utilizing this open source Spark feature on Databricks, is Unity C...
- 879 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @gandersen-codes , I recommend reaching out directly to Databricks support by filing a support ticket.
- 1 kudos
- 1489 Views
- 1 replies
- 1 kudos
Help Needed: Grants remaining for removed Groups and Service Principal
We have an issue in a Workspace which is managed by Terraform, a change went in to update the Group and Service Principal (SP) names but due to the internal ordering the Groups and SP were removed and replaced before the Grants were updated.If we now...
- 1489 Views
- 1 replies
- 1 kudos
- 1 kudos
We are experiencing a similar issue, except with the storage credential resource. We created some storage credentials using Terraform, but when trying to destroy them using Terraform they were ignored. So we decided to manually delete the storage cre...
- 1 kudos
- 1660 Views
- 1 replies
- 1 kudos
Why do we need different AWS Accounts ?
I am just going through the AWS Databricks Platform Administration course and I am curious to know about the Cloud Accounts used to setup DatabricksWhen we are setting up the Databricks, we are using AWS Account but we are not using that account sayi...
- 1660 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @HappySK, The AWS account you're using basically serves as the data plane - this is where your data lives, and the compute resources that process it (at least in the classical compute model) will be provisioned there as well. As part of the confi...
- 1 kudos
- 937 Views
- 1 replies
- 0 kudos
PAT Tokens access restrictions
Hi,We have a Databricks workspace on which we have disabled PAT's for now. Moving ahead we want the developers to use Personal Access Tokens but only for development purpose.As I know the current option to enable PAT's give access to the whole REST A...
- 937 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @paritoshsh, As per the current Databricks documentation, there is no direct way to add restrictions to Personal Access Tokens (PATs). When you create a PAT, it provides access to the entire Databricks REST API. However, you can restrict the gene...
- 0 kudos
- 3762 Views
- 1 replies
- 1 kudos
Resolved! Consequences of removing a workspace from a metastore in Azure Databricks
In the documentation (Enable a workspace for Unity Catalog - Azure Databricks | Microsoft Learn), it appears that I can remove a workspace from a metastore, and as long as the workspace has jobs that don't use tables, files, and models stored in any ...
- 3762 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @m997al , Based on the information provided, if you unlink a workspace from a metastore, the jobs in the workspace that do not depend on tables, files, and models stored in any catalog should still run. This is because the jobs are not using any l...
- 1 kudos
- 1392 Views
- 1 replies
- 1 kudos
Cluster failed to start
I am getting this error in my Partner Databricks Account and I had tried several methods to start the cluster. As i don't have access to console.aws.amazon.com/ec2. I was not able to check the details/logs in the ec2 instance. I am getting the follow...
- 1392 Views
- 1 replies
- 1 kudos
- 1 kudos
Here is a similar topic:https://community.databricks.com/t5/machine-learning/problem-with-spinning-up-a-cluster-on-a-new-workspace/m-p/29996To actually fix/analyse the issue, you need access to the EC2 console unfortunately. I assume someone in the ...
- 1 kudos
- 1658 Views
- 1 replies
- 1 kudos
Resolved! Service Principal for remote repository in workflow/job expiring token
I would like to create a databricks Job where the 'Run as' field is set to a ServicePrincipal. The Job points to notebooks stored in Azure DevOps.The step I've already performed are:I created the Service Principal and I'm now able to see it into the ...
- 1658 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @diego_poggioli, Unfortunately, there is no direct way to bypass the use of expiring tokens when accessing Azure DevOps. The Azure DevOps PAT is used as a security measure to ensure that only authorized users can access the resources, and it is de...
- 1 kudos
- 730 Views
- 1 replies
- 0 kudos
Ubuntu 18.4 EOL
Hi,last July 18th we were informed by Databricks that Ubuntu version 20.04 (operating system: Ubuntu 20.04.4 LTS) was going to be the only certified and supported Ubuntu version for the 10.4 runtime cluster we use. We have been experiencing some issu...
- 730 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @SanderJvanDijk, it's unclear whether the issues you're experiencing with Databricks libraries are directly caused by the new Ubuntu version (20.04.4 LTS) that Databricks pushed. The provided information indicates that Databricks has been continu...
- 0 kudos
- 2660 Views
- 2 replies
- 0 kudos
Resolved! Leverage Azure PIM with DataBricks with Contributor role privilege
We are trying to leverage Azure PIM. This works great for most things, however; we've run into a snag. We want to limit the contributor role to a group and only at the resource group level, not subscription. We wish to elevate via PIM. This will ...
- 2660 Views
- 2 replies
- 0 kudos
- 0 kudos
Thanks - think we were originally overthinking this.We determined we were doing this correctly, the user just needed to switch to 'groups' within PIM to request elevation of permissions. The larger issue is actually the 40 min user provisioning cycl...
- 0 kudos
