- 7245 Views
- 3 replies
- 2 kudos
How to bind a User assigned Managed identity to Databricks to access external resources?
Is there a way to bind a user assigned managed identity to Databricks? We want to access some SQL DBs, Redis cache from our Spark code running on Databricks using Managed Identity instead of Service Principals and basic authentication.As of today, Da...
- 7245 Views
- 3 replies
- 2 kudos
- 2 kudos
@Carpender correcting my comment above, Databricks assigned Managed Identity is working and we are able to access but as stated in the original question we are looking for authorization using User Assigned Managed Identity (UAMI). With UAMI we cannot...
- 2 kudos
- 5841 Views
- 1 replies
- 0 kudos
How to setup service principal to assing account-level groups to workspaces using terraform
Based on best practices, we have set up SCIM provisioning using Microsoft Entra ID to synchronize Entra ID groups to our Databricks account. All workspaces have identity federation enabled.However, how should workspace administrators assign account-l...
- 5841 Views
- 1 replies
- 0 kudos
- 0 kudos
Have you tried giving Manager role on the group to the service principal which is workspace admin? Once you do this you may be able to use the settings to In workspace context, adding account-level group to a workspace in databricks_permission_assig...
- 0 kudos
- 2133 Views
- 1 replies
- 0 kudos
Resolved! Instances are not being terminated in time (extra AWS costs)
For a few days we have been trying to figure out why our AWS costs suddenly went up around March 20th, and we just found the answer: the EC2 instances are left in an unterminated state for a couple of minutes at the end of each run! This is a very se...
- 2133 Views
- 1 replies
- 0 kudos
- 0 kudos
Nevermind, this was actually due to a reservation that expired. 臘
- 0 kudos
- 2921 Views
- 2 replies
- 1 kudos
Error: Node.js SQL Driver auth using M2M
I am trying to follow the documentation in order to establish a M2M authentication through the Node.js SQL Driver. I am however having issues as it results in the following error message. What is it that I am not seeing here?Error msg:{"level":"info"...
- 2921 Views
- 2 replies
- 1 kudos
- 1 kudos
Not sure if you're still suck on this but I had the same issue and managed to resolve it by adding azureTenantId: <my-tenant-id> and useDatabricksOAuthInAzure: true in the client connection options. Hope this helps!
- 1 kudos
- 4903 Views
- 2 replies
- 0 kudos
Setup unity catalog external location to minio
We have a minio server running in Azure that we have connected to the spark clusters directly. As we move to unity catalog, we would like to make the data stored in our minio servers accessible as an external location in Azure Databricks account via ...
- 4903 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Retired_mod ,I have a server on Azure that supports the S3 protocol, and I am trying to follow these instructions in order to use Unity on Azure DataBricks with it. I am not sure about this part of your reply:Set the Spark configuration values i...
- 0 kudos
- 7113 Views
- 3 replies
- 3 kudos
Operations on Unity Catalog take too long
Hi!We are currently PoC-ing Databricks with Unity Catalog on AWS but it seems there are some issues.Creating a database in an existing (unity) catalog takes over 10 minutes. Creating an external table on top of an existing delta table (CREATE TABLE m...
- 7113 Views
- 3 replies
- 3 kudos
- 3 kudos
PS: Apparently I'm not allowed to use the world H E A L T H (without spaces) in my reply (The message body contains H e a l t h, which is not permitted in this community. Please remove this content before sending your post.)
- 3 kudos
- 2979 Views
- 1 replies
- 0 kudos
How to configure an AWS so that workspace databricks can only access the s3 acces point using VPC
My team requires a configuration so that a databricks workspace can connect to aws s3 access point through VPC and that other databricks workspaces cannot access it if they are not within the route table.I have searched online, but I have only found ...
- 2979 Views
- 1 replies
- 0 kudos
- 3905 Views
- 1 replies
- 0 kudos
Access Logs
How can I check the timing when a particular AAD group was given access to a particular schema in a unity catalogue?Is there any API I can call to get this logs?
- 3905 Views
- 1 replies
- 0 kudos
- 4861 Views
- 4 replies
- 0 kudos
Use Unity External Location with full paths in delta_log
I have an external delta table in unity catalog (let's call it mycatalog.myschema.mytable) that only consists of a `_delta_log` directory that I create semi-manually, with the corresponding JSON files that define it. The JSON files point to parquet f...
- 4861 Views
- 4 replies
- 0 kudos
- 0 kudos
I suggest you look at something else than UC for such cases. I also wonder if delta lake is the right format.
- 0 kudos
- 3722 Views
- 1 replies
- 1 kudos
GCP - (DWH) Cluster Start-up Delayed - Failing to start
I face the issue that my fresh new Databricks workspace is not capable to start any cluster."Cluster Start-up Delayed. Please wait while we continue to try and start the cluster. No action is required from you."After 1830 seconds (30,5 minutes) the w...
- 3722 Views
- 1 replies
- 1 kudos
- 7131 Views
- 2 replies
- 0 kudos
authorizationfailure on ls fs on mount point files
One of our users has as of last week started seeing an authorization failure when he tries to list the files in the Azure storage account using Databricks Cli or Databricks API(using Python). He can list files on the Databricks portal or through the ...
- 7131 Views
- 2 replies
- 0 kudos
- 0 kudos
Have you checked the list of allowed ip addresses that are set for the Storage account in Azure? Is user using VPN or internal network, we might need to confirm if the network where the user is trying to list is set as allowed.
- 0 kudos
- 2438 Views
- 1 replies
- 0 kudos
User not authorised to copy files to dbfs
Hi,I'm trying to use a service principal to copy files to dbfs using the command line "databricks fs cp <source> <target>" but get back "User not authorised". I configured the authentication with PAT token and it is successful as I can deploy and lau...
- 2438 Views
- 1 replies
- 0 kudos
- 0 kudos
In Databricks, data access permissions are often managed separately from workspace permissions. For DBFS, access control is typically managed through the underlying cloud storage (Azure Blob Storage, S3, etc.). The service principal needs to have the...
- 0 kudos
- 3999 Views
- 0 replies
- 0 kudos
Send formatted html email from email distribution address
Hi, I have created an email distribution list "#MyList@mycompany.com". In the RShiny world I was able to send emails by a) getting the IP of the server I was sending the emails from and b) whitelisting that IP address within my company's SMTP Relay r...
- 3999 Views
- 0 replies
- 0 kudos
- 2675 Views
- 0 replies
- 0 kudos
Delta Sharing resulting in Bad_Request
Hi All,Recipeint encountering an issue while trying to access my organizational data (providers data) in a Delta Sharing scenario(databricks to databricks), and I'm hoping to get some guidance on how to resolve it. Here is the error message recipient...
- 2675 Views
- 0 replies
- 0 kudos
- 2517 Views
- 0 replies
- 0 kudos
Azure Databricks with standard private link (only one Databricks authentication workspace)?
We have successfully set up Azure Databricks with standard private link (front-end and back-end).The front-end uses the authentication workspace as prescribed in the documentation.Suppose we use a "custom DNS" for the configuration below. Can we set...
- 2517 Views
- 0 replies
- 0 kudos
