- 3473 Views
- 3 replies
- 5 kudos
Resolved! Move to 100% Serverless
Hi all,A few questions about the upcoming transition to 100% serverless, if anyone has any info that would be great!When will the move to serverless occur? I understand from 1st July (today) but has anyone seen a roadmap?What will the move to serverl...
- 3473 Views
- 3 replies
- 5 kudos
- 5 kudos
Hi, so our Databricks contact just assured us the following, after we asked about this issue:Databricks is officially (but won’t be GA in every region till end of July) 100% Serverless OPTIONAL.We understand many of our customers have begged for 100%...
- 5 kudos
- 1914 Views
- 1 replies
- 0 kudos
Resolved! Unity Catalog Pandas on Spark Limitation
According to Databricks UC Documentation, below are the some of the limitations on Shared Mode Cluster.1. In Databricks Runtime 13.3 LTS and above, Python scalar UDFs and Pandas UDFs are supported. Other Python UDFs, including UDAFs, UDTFs, and Panda...
- 1914 Views
- 1 replies
- 0 kudos
- 0 kudos
@RamlaSuhra, UDFs on Unity Catalog is a feature that, at the current moment is still on the Public Preview stage. This means that the development has yet not finished. UDFs can be used on DBR 13.3 and above, UDAFs are already available for DBR 15.2 ...
- 0 kudos
- 1225 Views
- 2 replies
- 0 kudos
New Users don't receive onboarding email
When I create a new user in Databricks, the new user does not receive their onboarding email. It is not in their junkmail, deleted items or in their inbox.However, when I reset that user's password, they do receive the password reset link, and are a...
- 1225 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @billraper , I'm sorry to hear about the trouble. Would you mind sharing more about whether this is happening with a community portal profile or with a product profile? Please share the link to the profile with which you're experiencing this issue...
- 0 kudos
- 931 Views
- 1 replies
- 0 kudos
cannot login to account management
HiI am not able to login to account management (https://accounts.cloud.databricks.com), It somehow reninforce SSO, cannot login with username and password.
- 931 Views
- 1 replies
- 0 kudos
- 1760 Views
- 1 replies
- 1 kudos
Coarse-grained access management for jobs
Are there any perspectives in Databricks' roadmap for enabling coarse-grained access management for jobs?Currently, access to jobs has to be managed on a job by job basis: https://docs.databricks.com/en/security/auth-authz/access-control/index.html#j...
- 1760 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Retired_mod, thanks for your reply.A more mature access management concept in Databricks would be definitely terrific. I understand it's not entirely along the AI-lines that Databricks is pushing hard currently, but it would improve a lot the pla...
- 1 kudos
- 6522 Views
- 3 replies
- 3 kudos
Is there a setting which restricts users from Creating Job and Pipeline?
as far i know currently ((as of 03-25-2024) databricks don't any workspace admin settings option to restrict users from creating a workflow/job or delta pipelines. Here is the use case for it Example: you have 3 tier landscape Dev, Qa and Prod.It is ...
- 6522 Views
- 3 replies
- 3 kudos
- 3 kudos
I notice that a separate discussion overlaps with the OPs issue: https://community.databricks.com/t5/data-engineering/restricting-workflow-creation-and-implementing-approval/td-p/4336@Retired_mod do you have a mechanism for clustering discussions fo...
- 3 kudos
- 1638 Views
- 0 replies
- 0 kudos
Can you all share your experiences on rolling out new features in Workspaces managed by you as Admin
As Data Platform Admins, do you follow some standard process or self service approach towards rolling out new features in your workspaces? Is the process automated? How is the testing done? Please share your thoughts. Given UC is introducing new feat...
- 1638 Views
- 0 replies
- 0 kudos
- 8149 Views
- 2 replies
- 2 kudos
Get Azure Databricks Account ID
Hi everyone,Is it possible with Terraform or Azure CLI or any other not manual method to get the value for Azure Databricks Account ID and not to use manual method as is described here - https://learn.microsoft.com/en-us/azure/databricks/administrati...
- 8149 Views
- 2 replies
- 2 kudos
- 2 kudos
I am also looking for a solution to this. The path suggested by @Retired_mod does not work! When I run the proposed az cli, this is what I get:```$> az databricks workspace show --resource-group $my_rg --name $my_ws --query 'id'"/subscriptions/64e..<...
- 2 kudos
- 3624 Views
- 4 replies
- 0 kudos
Unity catalog and metadata to backup site
Hi ,I am looking for Databricks DR(Disaster recovery) site creation. My primary setup is in west us and my primary data bricks created in primary site with unity catalog enable. with same meta data i want bring up the Databricks in my secondary sid...
- 3624 Views
- 4 replies
- 0 kudos
- 0 kudos
As of now UC does not have DR setting, this is currently in our prioritized roadmap but no ETA is currently available. Some customers host an external metastore and replicate it across regions to achieve this.
- 0 kudos
- 827 Views
- 0 replies
- 0 kudos
How are teams platform admin teams managing "allow list" for libraries feature UC?
We have so many team using maven libraries and are in the process of UC migration. These maven coordinates need to be added to "allow list" before they can be used in clusters. What is the standard process followed by admin teams for this feature? Do...
- 827 Views
- 0 replies
- 0 kudos
- 1014 Views
- 0 replies
- 0 kudos
Qa Environments
What is the best practice to do a paralele test for many chaches on the same ETL?
- 1014 Views
- 0 replies
- 0 kudos
- 1993 Views
- 0 replies
- 0 kudos
link databrick to AWS GOV cloud
Does anyone know how to link databricks to AWS GOV cloud? I subscribed to databricks and created databricks account from the link in Marketplace. When I go to databricks account console the URL doesn't take me to the URL reference in the documentatio...
- 1993 Views
- 0 replies
- 0 kudos
- 1026 Views
- 0 replies
- 0 kudos
Databricks AWS ACLs requirement
I'm going through the documentation in databricks for using existing VPC requirements. For the section that covers ACLs, what's the need to have this inbound rule: " ALLOW ALL from Source 0.0.0.0/0. This rule must be prioritized."https://docs.databri...
- 1026 Views
- 0 replies
- 0 kudos
- 2830 Views
- 3 replies
- 0 kudos
Unity Catalog read issue
Hello,Our company is POCing the Unity Catalog with Azure as provider.We have 2 subscriptions that contains 1 databricks each and 1 ADLS GEN2 each.Initially we have the default `hive_metastore` connected to the ADLS GEN2. I've created a secret scope a...
- 2830 Views
- 3 replies
- 0 kudos
- 0 kudos
Hello,The UC catalog container is sitting in Sub B.Basically we have a Spoke and Hub configuration for each Subscriptions. Each Subscriptions can access to any ressource inside it own subscription with some PE. But to access other Subscription ressou...
- 0 kudos
- 2096 Views
- 1 replies
- 0 kudos
Using JARs from Google Cloud Artifact Registry in Databricks for Job Execution
We have our CI/CD pipelines set up in Google Cloud using Cloud Build, and we are publishing our artifacts to a private repository in Google Cloud's Artifact Registry. I want to use these JAR files to create and run jobs in Databricks.However, when I ...
- 2096 Views
- 1 replies
- 0 kudos
- 0 kudos
To integrate your CI/CD pipeline with Databricks, you have a couple of options:1. Using Artifact Registry as an intermediary: Currently, Databricks does not directly support Google Artifact Registry URLs. However, you can use an intermediate storage ...
- 0 kudos
