- 1257 Views
- 3 replies
- 0 kudos
Permission denied using patchelf
Hello all I would like to run a Python script on a Shared cluster: The Python script tries to do, under the hood, a call to `patchelf` utility, in order to set a r-path. Something along those lines:execute('patchelf',['--set-rpath', rpath, lib])The p...
- 1257 Views
- 3 replies
- 0 kudos
- 0 kudos
I basically followed this tutorial https://learn.microsoft.com/en-gb/azure/databricks/data-governance/unity-catalog/get-started and it seemed to be working, so I guess Unity Catalog is enabled?
- 0 kudos
- 2201 Views
- 1 replies
- 0 kudos
Why does use of Azure SSO require Databricks PAT enabled ?
My org uses Databricks and SSO. We are keen to disable the use of PAT but have noticed that when it's disabled, we're not able to use SSO. May I ask why does SSO have a dependency on PATs [arguably they are two distinct authentication methods] ?Also,...
- 2201 Views
- 1 replies
- 0 kudos
- 0 kudos
My org uses Databricks and SSO. We are keen to disable the use of PAT but have noticed that when it's disabled, we're not able to use SSO. May I ask why does SSO have a dependency on PATs [arguably they are two distinct authentication methods] ?Also,...
- 0 kudos
- 1684 Views
- 0 replies
- 0 kudos
Delete Azure Databricks Workspace resource but reference remains in Account Console
As the title says. I have deleted both the Azure Databricks Workspace resource and the resource group it was located in. However, I can still see the workspace in the account console. Also, it appears in the response from GET https://accounts.azureda...
- 1684 Views
- 0 replies
- 0 kudos
- 3996 Views
- 2 replies
- 0 kudos
Resolved! Call an Azure Function App with Access Restrictions from a Databricks Workspace
Hello,As the title says, I am trying to call an function from an Azure Function App configured with access restrictions from a python notebook in my Databricks workspace. The Function App resource is in a different subscription as the Databricks work...
- 3996 Views
- 2 replies
- 0 kudos
- 0 kudos
Update : Problem was fixed ! The key was to set an VNET rule in the access restriction, giving access directly to the subnets used by Databricks.It seems like for Microsoft to Microsoft connections, the IP addresses are not used, so adding the IP ran...
- 0 kudos
- 1710 Views
- 0 replies
- 0 kudos
Azure Devops repos access
I have a Databricks setup, where the users and their permissions are handled in Microsoft Azure using AD groups and then provisioned (account level) using a provisioning connector to Databricks. The code repositories are in Azure Devops where users a...
- 1710 Views
- 0 replies
- 0 kudos
- 3894 Views
- 3 replies
- 1 kudos
How to bind a User assigned Managed identity to Databricks to access external resources?
Is there a way to bind a user assigned managed identity to Databricks? We want to access some SQL DBs, Redis cache from our Spark code running on Databricks using Managed Identity instead of Service Principals and basic authentication.As of today, Da...
- 3894 Views
- 3 replies
- 1 kudos
- 1 kudos
@Carpender correcting my comment above, Databricks assigned Managed Identity is working and we are able to access but as stated in the original question we are looking for authorization using User Assigned Managed Identity (UAMI). With UAMI we cannot...
- 1 kudos
- 4890 Views
- 1 replies
- 0 kudos
How to setup service principal to assing account-level groups to workspaces using terraform
Based on best practices, we have set up SCIM provisioning using Microsoft Entra ID to synchronize Entra ID groups to our Databricks account. All workspaces have identity federation enabled.However, how should workspace administrators assign account-l...
- 4890 Views
- 1 replies
- 0 kudos
- 0 kudos
Have you tried giving Manager role on the group to the service principal which is workspace admin? Once you do this you may be able to use the settings to In workspace context, adding account-level group to a workspace in databricks_permission_assig...
- 0 kudos
- 1369 Views
- 1 replies
- 0 kudos
Resolved! Instances are not being terminated in time (extra AWS costs)
For a few days we have been trying to figure out why our AWS costs suddenly went up around March 20th, and we just found the answer: the EC2 instances are left in an unterminated state for a couple of minutes at the end of each run! This is a very se...
- 1369 Views
- 1 replies
- 0 kudos
- 0 kudos
Nevermind, this was actually due to a reservation that expired. 臘
- 0 kudos
- 2939 Views
- 5 replies
- 0 kudos
System.billing.usage table - cannot match job_id from databricks api/UI
Hello, I have multiple continuous jobs that are running for many days (Kafka stream), however querying System.billing.usage table by job_id from UI or databricks job api not return any results for those jobs.1. What is the reason behind that?2. If I ...
- 2939 Views
- 5 replies
- 0 kudos
- 0 kudos
Hello, you are right, apologize for my misunderstanding as you have mentioned the job id persist, is the job run id the one that will be changed. So your issue is that as the job is continuing running the information of that run is not showing in the...
- 0 kudos
- 1504 Views
- 2 replies
- 0 kudos
Error: Node.js SQL Driver auth using M2M
I am trying to follow the documentation in order to establish a M2M authentication through the Node.js SQL Driver. I am however having issues as it results in the following error message. What is it that I am not seeing here?Error msg:{"level":"info"...
- 1504 Views
- 2 replies
- 0 kudos
- 0 kudos
Not sure if you're still suck on this but I had the same issue and managed to resolve it by adding azureTenantId: <my-tenant-id> and useDatabricksOAuthInAzure: true in the client connection options. Hope this helps!
- 0 kudos
- 2752 Views
- 1 replies
- 0 kudos
How to get Workspace name with workspaceId?
I have an AWS Managed Databricks instance. I am trying to get a workspace name with workspace ID. Thank you very much for your time and assistance.
- 2752 Views
- 1 replies
- 0 kudos
- 0 kudos
I have an AWS Managed Databricks instance. I am trying to get a workspace name with workspace ID. Thank you very much for your time and assistance.
- 0 kudos
- 1450 Views
- 0 replies
- 0 kudos
It's possible to sync account group/user to workspace without the need to do it manually?
I am using Databricks SCIM for my Databricks Account, so when i add a user or group in the SCIM connector, the user or group its created in Databricks Account. After this, i need to manually assign the user/group to the workspaces. My boss wants to o...
- 1450 Views
- 0 replies
- 0 kudos
- 859 Views
- 0 replies
- 1 kudos
Delta Live Table pipeline steps explanation
Does anyone have documentation on what is actually occurring in each of these steps?Creating update Waiting for resourcesInitializingSetting up tablesRendering graphFor example, what is the difference between initializing and setting up tables? I am ...
- 859 Views
- 0 replies
- 1 kudos
- 2270 Views
- 2 replies
- 0 kudos
Setup unity catalog external location to minio
We have a minio server running in Azure that we have connected to the spark clusters directly. As we move to unity catalog, we would like to make the data stored in our minio servers accessible as an external location in Azure Databricks account via ...
- 2270 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Retired_mod ,I have a server on Azure that supports the S3 protocol, and I am trying to follow these instructions in order to use Unity on Azure DataBricks with it. I am not sure about this part of your reply:Set the Spark configuration values i...
- 0 kudos
- 4402 Views
- 3 replies
- 2 kudos
Operations on Unity Catalog take too long
Hi!We are currently PoC-ing Databricks with Unity Catalog on AWS but it seems there are some issues.Creating a database in an existing (unity) catalog takes over 10 minutes. Creating an external table on top of an existing delta table (CREATE TABLE m...
- 4402 Views
- 3 replies
- 2 kudos
- 2 kudos
PS: Apparently I'm not allowed to use the world H E A L T H (without spaces) in my reply (The message body contains H e a l t h, which is not permitted in this community. Please remove this content before sending your post.)
- 2 kudos
