- 4278 Views
- 2 replies
- 1 kudos
Resolved! Compute configuration : single user with service principal of azure data datafactory ?
Is it possible to have the service principal (ID) of an Azure data factory as the Single user access on an databricks cluster ?Reason I'm asking is because we are starting to use unity catalog , but would still have the need to execute stored procedu...
- 4278 Views
- 2 replies
- 1 kudos
- 1 kudos
Yes, this is possible. First, create a new service principal in Azure or use an existing one. This could be either a managed identity from Azure Data Factory or a manually created service principal in Microsoft Entra ID (formerly Azure AD). Next, in ...
- 1 kudos
- 1005 Views
- 2 replies
- 0 kudos
How do I identify who triggered my Databricks job?
How can I identify who triggered my Databricks job? The Databricks job is running via a service principal. One of my runs initially failed, but a repair occurred 30 minutes later, causing the job to enter a successful state. I would like to determine...
- 1005 Views
- 2 replies
- 0 kudos
- 0 kudos
thanks for sharing the infocan you please share the audit logs query so I can pass.
- 0 kudos
- 1413 Views
- 3 replies
- 1 kudos
Resolved! How to add existing recipient to existing delta share
I am facing an issue while adding a recipient to Delta Share using Terraform. The owner of my recipient is a group, not an individual user. I'm running this Terraform script using a service principal member of that group. However, I'm encountering th...
- 1413 Views
- 3 replies
- 1 kudos
- 1 kudos
I was able to fix the issue. The problem was that the service principal I was using didn’t have the correct permissions assigned
- 1 kudos
- 3208 Views
- 10 replies
- 0 kudos
Resolved! Python function "go to definition" and "peek definition" do not work
When using notebooks with python in Databricks I would really like to easily see the defintion of the functions I am using within the editor. Which the "Go to definition (F12)" and "Peek definition" options when right clicking on the functions will h...
- 3208 Views
- 10 replies
- 0 kudos
- 0 kudos
This has now been resolved and is working as expected. Do not know why or how, but something has changed that made it work
- 0 kudos
- 4464 Views
- 5 replies
- 2 kudos
Adhoc workflows - managing resource usage on shared clusters
We run a shared cluster that is used for general purpose adhoc analytics, which I assume is a relatively common use case to try to keep costs down. However, the technical experience of users of this cluster varies a lot, so we run into situations whe...
- 4464 Views
- 5 replies
- 2 kudos
- 2 kudos
Hi, @JameDavi_51481 , were you able to figure something out?Planning a Databricks migration and realized we might need something similar too.
- 2 kudos
- 728 Views
- 2 replies
- 0 kudos
Sporadic HTTP failure with SQL Serverless (bug?)
Our SQL Serverless installation has sporadic failures to our blob container in Azure. The blob container is locked down to a vnet, and we are using the private endpoint to enable serverless access. It will work fine for several hours, and then show...
- 728 Views
- 2 replies
- 0 kudos
- 0 kudos
I've confirmed all of that. This seems like an AI generated response. It seems more likely that Databricks rolled out a feature a week ago that is causing instability in the serverless warehouses. Any other specific things to check would be apprec...
- 0 kudos
- 1100 Views
- 1 replies
- 0 kudos
I want to create custom tag in cluster policy so that clusters created using that policy get those
"I want to create custom tags in a cluster policy so that clusters created using this policy will automatically include those tags for billing purposes. Consider the following example:"cluster_type": {"type": "fixed","value": "all-purpose"},"custom_t...
- 1100 Views
- 1 replies
- 0 kudos
- 0 kudos
Are you having any issue while running this code in the policy?
- 0 kudos
- 1331 Views
- 1 replies
- 0 kudos
Databricks Serverless best practices
Hi All, We are configuring a Databricks serverless that adjusts according to the workload type,like choosing different cluster sizes such as extra small ,small ,large etc, and auto scale option.We're also looking at the average time it takes to compl...
- 1331 Views
- 1 replies
- 0 kudos
- 0 kudos
You can refer to our Serverless Compute Best practices: https://docs.databricks.com/en/compute/serverless/best-practices.htmlIf you refer to the Serverless Warehouses you can refer to https://docs.databricks.com/en/compute/sql-warehouse/warehouse-beh...
- 0 kudos
- 725 Views
- 1 replies
- 0 kudos
Manage Account option
HiI have created a premium databricks workspace on my azure free trial account and also have the global administrator role on my azure acount. I have setup all the necessary configurations like by providing the role of storage data blob contributor t...
- 725 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @SALAHUDDINKHAN, If you are unable to see the "Manage Account" option, it is likely that you do not have the necessary account admin privileges. Please ensure you have the required permissions indicated here: https://learn.microsoft.com/en-us/a...
- 0 kudos
- 12191 Views
- 2 replies
- 1 kudos
Resolved! Any way to move the unity catalog to a new external storage location?
Dear Databricks CommunityThe question is about changing an existing unity catalog to a new storage location. For example: With an existing unity catalog (i.e. catalog1) includeing schemas and volumes. The catalog is based on an external location (i....
- 12191 Views
- 2 replies
- 1 kudos
- 1 kudos
https://docs.databricks.com/ja/sql/language-manual/sql-ref-syntax-ddl-alter-location.html
- 1 kudos
- 1398 Views
- 3 replies
- 1 kudos
cloud_infra_costs
I was looking at the system catalog and realized that there is an empty table called cloud_infra_costs. Could you tell me what is this for and why it is empty?
- 1398 Views
- 3 replies
- 1 kudos
- 1 kudos
Thanks for replying. This makes sense. Any idea why it is empty and what to do to populate this?
- 1 kudos
- 3706 Views
- 5 replies
- 2 kudos
Ingress/Egress private endpoint
Hello ,We have configured our Databricks environment with private endpoint connections injected into our VNET, which includes two subnets (public and private). We have disabled public IPs and are using Network Security Groups (NSGs) on the subnet, as...
- 3706 Views
- 5 replies
- 2 kudos
- 2 kudos
@Fkebbati First, traffic cost in Azure are not reported as a separate Resource Type, but appended to main resource causing the traffic. If you want to distinguish them use for instance Service Name. In this case traffic cost is appended to Databricks...
- 2 kudos
- 1657 Views
- 3 replies
- 0 kudos
Databricks (GCP) Cluster not resolving Hostname into IP address
we have #mongodb hosts that must be resolved to private internal loadbalancer ips ( of another cluster ), and that we are unable to add host aliases in the Databricks GKE cluster in order for the spark to be able to connect to a mongodb and resolve t...
- 1657 Views
- 3 replies
- 0 kudos
- 0 kudos
Also found this - https://community.databricks.com/t5/data-engineering/why-i-m-getting-connection-timeout-when-connecting-to-mongodb/m-p/14868
- 0 kudos
- 2689 Views
- 3 replies
- 0 kudos
Data leakage risk happened when we use the Azure Databricks workspace
Context:We are utilizing an Azure Databricks workspace for data management and model serving within our project, with delegated VNet and subnets configured specifically for this workspace. However, we are consistently observing malicious flow entries...
- 2689 Views
- 3 replies
- 0 kudos
- 0 kudos
Hello everyone! We have worked with our security team, Microsoft, and other customers who have seen similar log messages. This log message is very misleading, as it appears to state that the malicious URI was detected within your network — this would...
- 0 kudos
- 893 Views
- 1 replies
- 0 kudos
Unable to add a databricks permission to existing policy
Hi, We're using databricks provider v1.49.1 to manager our Azure databricks cluster and other resources. Having an issue setting permissions with the databricks terraform resource "databricks_permissions" where the error indicates that the clust...
- 893 Views
- 1 replies
- 0 kudos
- 0 kudos
Is this cluster policy a custom policy? If you try for testing purposes to modify it in the UI does it allows you to
- 0 kudos
