- 5704 Views
- 2 replies
- 0 kudos
authorizationfailure on ls fs on mount point files
One of our users has as of last week started seeing an authorization failure when he tries to list the files in the Azure storage account using Databricks Cli or Databricks API(using Python). He can list files on the Databricks portal or through the ...
- 5704 Views
- 2 replies
- 0 kudos
- 0 kudos
Have you checked the list of allowed ip addresses that are set for the Storage account in Azure? Is user using VPN or internal network, we might need to confirm if the network where the user is trying to list is set as allowed.
- 0 kudos
- 1884 Views
- 1 replies
- 0 kudos
User not authorised to copy files to dbfs
Hi,I'm trying to use a service principal to copy files to dbfs using the command line "databricks fs cp <source> <target>" but get back "User not authorised". I configured the authentication with PAT token and it is successful as I can deploy and lau...
- 1884 Views
- 1 replies
- 0 kudos
- 0 kudos
In Databricks, data access permissions are often managed separately from workspace permissions. For DBFS, access control is typically managed through the underlying cloud storage (Azure Blob Storage, S3, etc.). The service principal needs to have the...
- 0 kudos
- 3143 Views
- 0 replies
- 0 kudos
Send formatted html email from email distribution address
Hi, I have created an email distribution list "#MyList@mycompany.com". In the RShiny world I was able to send emails by a) getting the IP of the server I was sending the emails from and b) whitelisting that IP address within my company's SMTP Relay r...
- 3143 Views
- 0 replies
- 0 kudos
- 1663 Views
- 0 replies
- 0 kudos
Delta Sharing resulting in Bad_Request
Hi All,Recipeint encountering an issue while trying to access my organizational data (providers data) in a Delta Sharing scenario(databricks to databricks), and I'm hoping to get some guidance on how to resolve it. Here is the error message recipient...
- 1663 Views
- 0 replies
- 0 kudos
- 1607 Views
- 0 replies
- 0 kudos
Azure Databricks with standard private link (only one Databricks authentication workspace)?
We have successfully set up Azure Databricks with standard private link (front-end and back-end).The front-end uses the authentication workspace as prescribed in the documentation.Suppose we use a "custom DNS" for the configuration below. Can we set...
- 1607 Views
- 0 replies
- 0 kudos
- 1494 Views
- 1 replies
- 0 kudos
Unity catalog metastore and databricks workspace provisioning
Hi,I am using terraform to create a databricks metastore using this.Now the issue is that I want to use this metastore reference in another terraform project where I create the databricks workspaces but there is no data source for metastore. Can any...
- 1494 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi,I am using terraform to create a databricks metastore using this.Now the issue is that I want to use this metastore reference in another terraform project where I create the databricks workspaces but there is no data source for metastore. Can any...
- 0 kudos
- 3267 Views
- 0 replies
- 0 kudos
Databricks Rest APIs CORS Issue
Hello Team,We are currently integrating Databricks Rest APIs into our in-house application for managing access permissions. While testing with curl and Postman, we've successfully accessed certain APIs like listing cluster permission. However, we're ...
- 3267 Views
- 0 replies
- 0 kudos
- 4746 Views
- 0 replies
- 0 kudos
Workflow (Job) Cluster Permission Management
Hello Team,I understand that as the Job Owner, they can grant additional permissions to other users to manage/run/view the job.If "Can Manage" permission is given to the other users, that user can Edit the job including the Run-As parameter to themse...
- 4746 Views
- 0 replies
- 0 kudos
- 1495 Views
- 0 replies
- 0 kudos
Cluster with shared access mode cannot query metastore
Hi, I have created a new UC enabled metastore using an Azure storage account and container connected to a Databricks workspace using an access connector. At first glance everything seems to work. I encounter a problem, however, when I try to query UC...
- 1495 Views
- 0 replies
- 0 kudos
- 3180 Views
- 1 replies
- 0 kudos
PyPI cluster libraries failing to get installed
Hi all,In my cluster, some of the PyPI cluster libraries started failing to get installed. It is weird because some of them get installed and some of are constantly failing. In every failed one, the error message is the same (just a package name is d...
- 3180 Views
- 1 replies
- 0 kudos
- 0 kudos
I've had this issue myself. What ended up to be the problem is I had windows line endings in my .sh script. You need to convert them to Linux line endings.
- 0 kudos
- 16622 Views
- 1 replies
- 1 kudos
How to Know DBU consumption in azure databricks ?
In Azure portal - Billing we can get the COST but how to know How much DBU is consumed ?
- 16622 Views
- 1 replies
- 1 kudos
- 1 kudos
In Azure portal - Billing we can get the COST but how to know How much DBU is consumed ?
- 1 kudos
- 4199 Views
- 4 replies
- 1 kudos
Resolved! not able to see access schema inside system catalog
Hi,I am account admin role but not able to see access schema , I want to query audit log table
- 4199 Views
- 4 replies
- 1 kudos
- 1 kudos
Done, needed to enable the system schema -- Run this command in notebook- to get list of available schema %shcurl -v -X GET -H "Authorization: Bearer <PAT Token>" "https://adb-<xxx>.azuredatabricks.net/api/2.0/unity-catalog/metastores/<metastore-id>...
- 1 kudos
- 2842 Views
- 2 replies
- 0 kudos
Deleting MLflow runs programmatically
I would like to programmatically delete some MLflow runs based on a given run id. I am interested in the best practices on how to do this in Databricks workspaces.I know that MLflow cli has gc command which seems quite useful since it also deletes as...
- 2842 Views
- 2 replies
- 0 kudos
- 0 kudos
Do we have any updates on this? i'm using databricks and I have the same MlflowException
- 0 kudos
- 2502 Views
- 0 replies
- 0 kudos
User Provisioning ( SCIM for OKTA)
Hello Databricks I wanted to ask a couple questions regarding switching SSO from onelogin to OKTA and turning on user provisioning. We have a total of 4 workspaces ( 1 sandbox , 2 dev and 1 prod) within our account. We have unified login enabled for ...
- 2502 Views
- 0 replies
- 0 kudos
- 2502 Views
- 1 replies
- 1 kudos
SQL warehouse realtime monitoring
A few days ago, our SQL warehouse had been back pressure and I knew that issue a hour later.I really wanna know how to get status of databricks SQL warehouse such as using DataDog agent not using web UI.
- 2502 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi There! Is there any way to integrate Datadog with Databricks SQL Warehouse? I only found documentation related to clusters that seem to be different than SQL Warehouse in Databricks. However, I tried to add a global init script for Datadog integra...
- 1 kudos
