Hi,I am using terraform to create a databricks metastore using this.Now the issue is that I want to use this metastore reference in another terraform project where I create the databricks workspaces but there is no data source for metastore. Can any...
Hello Team,We are currently integrating Databricks Rest APIs into our in-house application for managing access permissions. While testing with curl and Postman, we've successfully accessed certain APIs like listing cluster permission. However, we're ...
Hello Team,I understand that as the Job Owner, they can grant additional permissions to other users to manage/run/view the job.If "Can Manage" permission is given to the other users, that user can Edit the job including the Run-As parameter to themse...
Hi, I have created a new UC enabled metastore using an Azure storage account and container connected to a Databricks workspace using an access connector. At first glance everything seems to work. I encounter a problem, however, when I try to query UC...
Hi all,In my cluster, some of the PyPI cluster libraries started failing to get installed. It is weird because some of them get installed and some of are constantly failing. In every failed one, the error message is the same (just a package name is d...
I've had this issue myself. What ended up to be the problem is I had windows line endings in my .sh script. You need to convert them to Linux line endings.
Hi,I am account admin role but not able to see access schema , I want to query audit log table
Done, needed to enable the system schema -- Run this command in notebook- to get list of available schema %shcurl -v -X GET -H "Authorization: Bearer <PAT Token>" "https://adb-<xxx>.azuredatabricks.net/api/2.0/unity-catalog/metastores/<metastore-id>...
I would like to programmatically delete some MLflow runs based on a given run id. I am interested in the best practices on how to do this in Databricks workspaces.I know that MLflow cli has gc command which seems quite useful since it also deletes as...
Do we have any updates on this? i'm using databricks and I have the same MlflowException
Hello Databricks I wanted to ask a couple questions regarding switching SSO from onelogin to OKTA and turning on user provisioning. We have a total of 4 workspaces ( 1 sandbox , 2 dev and 1 prod) within our account. We have unified login enabled for ...
A few days ago, our SQL warehouse had been back pressure and I knew that issue a hour later.I really wanna know how to get status of databricks SQL warehouse such as using DataDog agent not using web UI.
Hi There! Is there any way to integrate Datadog with Databricks SQL Warehouse? I only found documentation related to clusters that seem to be different than SQL Warehouse in Databricks. However, I tried to add a global init script for Datadog integra...
HelloI created a databricks job with a type "Python wheel". In the "Package name" field I assigned a python package which contains a dash in its name (see attach). The run of the job failed saying that could not import python package due to dash in t...
There you can see a complete template project with a python wheel task and modules. Please, follow the instructions for deployment.https://github.com/andre-salvati/databricks-template
I cant get the compute Tab to open.. just spins. I also cant start a cluster from a notebook. anyone else seeing this
Hi @Byron , Good Day! Can you please provide more details on what you are seeing or facing when you are accessing the compute section? Also please check with your workspace admin whether you have the correct cluster permission or not. You can refer...
Hi,When creating a new AWS Databricks workspace in account console manually, it appears that IAM Role ARN is mandatory parameter when configuring a Storage ConfigurationHowever in the Terraform databricks_mws_storage_configurations resource there is ...
Investigated this further... Terraform code to create the workspace resource "databricks_mws_workspaces" "databricks_ws" { provider = databricks.mws account_id = var.databricks_account_id aws_region = var.region workspace_name = "databricks-w...
As a workspace administrator, I'm finding that I cannot change the Service Principal Name (SPN) immediately after registering it through its application ID in Databricks. It appears that Databricks has linked the SPN name with its application ID and ...
Hi @Priyam1 , Hope you are doing well! You can change the display name of the Service Principal using the account SCIM APIs : https://docs.databricks.com/api/azure/account/accountserviceprincipals/patch Please let me know if this helps and leave a l...
I have a workspace deployed in AWS and need to read some secrets from AWS Secrets Manager in my notebook. I'm aware that there is no default process similar to Azure Key Vault, however I know that we can try to access it using boto3, but I'm stuck at...
Able to connect using personal access token and view the data in Hive; but when I publish to dashboard to share-point it results in :- PERMISSION_DENIED: You do not have permissions to autostart 1127-******-*********
