- 5210 Views
- 1 replies
- 0 kudos
PyPI cluster libraries failing to get installed
Hi all,In my cluster, some of the PyPI cluster libraries started failing to get installed. It is weird because some of them get installed and some of are constantly failing. In every failed one, the error message is the same (just a package name is d...
- 5210 Views
- 1 replies
- 0 kudos
- 0 kudos
I've had this issue myself. What ended up to be the problem is I had windows line endings in my .sh script. You need to convert them to Linux line endings.
- 0 kudos
- 7097 Views
- 4 replies
- 1 kudos
Resolved! not able to see access schema inside system catalog
Hi,I am account admin role but not able to see access schema , I want to query audit log table
- 7097 Views
- 4 replies
- 1 kudos
- 1 kudos
Done, needed to enable the system schema -- Run this command in notebook- to get list of available schema %shcurl -v -X GET -H "Authorization: Bearer <PAT Token>" "https://adb-<xxx>.azuredatabricks.net/api/2.0/unity-catalog/metastores/<metastore-id>...
- 1 kudos
- 4129 Views
- 2 replies
- 0 kudos
Deleting MLflow runs programmatically
I would like to programmatically delete some MLflow runs based on a given run id. I am interested in the best practices on how to do this in Databricks workspaces.I know that MLflow cli has gc command which seems quite useful since it also deletes as...
- 4129 Views
- 2 replies
- 0 kudos
- 0 kudos
Do we have any updates on this? i'm using databricks and I have the same MlflowException
- 0 kudos
- 3288 Views
- 0 replies
- 0 kudos
User Provisioning ( SCIM for OKTA)
Hello Databricks I wanted to ask a couple questions regarding switching SSO from onelogin to OKTA and turning on user provisioning. We have a total of 4 workspaces ( 1 sandbox , 2 dev and 1 prod) within our account. We have unified login enabled for ...
- 3288 Views
- 0 replies
- 0 kudos
- 3332 Views
- 1 replies
- 1 kudos
SQL warehouse realtime monitoring
A few days ago, our SQL warehouse had been back pressure and I knew that issue a hour later.I really wanna know how to get status of databricks SQL warehouse such as using DataDog agent not using web UI.
- 3332 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi There! Is there any way to integrate Datadog with Databricks SQL Warehouse? I only found documentation related to clusters that seem to be different than SQL Warehouse in Databricks. However, I tried to add a global init script for Datadog integra...
- 1 kudos
- 3094 Views
- 4 replies
- 0 kudos
Databricks job with a type "Python wheel" does not work if "Package name" contains dash
HelloI created a databricks job with a type "Python wheel". In the "Package name" field I assigned a python package which contains a dash in its name (see attach). The run of the job failed saying that could not import python package due to dash in t...
- 3094 Views
- 4 replies
- 0 kudos
- 0 kudos
There you can see a complete template project with a python wheel task and modules. Please, follow the instructions for deployment.https://github.com/andre-salvati/databricks-template
- 0 kudos
- 1836 Views
- 1 replies
- 1 kudos
having Problems starting a cluster on azure
I cant get the compute Tab to open.. just spins. I also cant start a cluster from a notebook. anyone else seeing this
- 1836 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Byron , Good Day! Can you please provide more details on what you are seeing or facing when you are accessing the compute section? Also please check with your workspace admin whether you have the correct cluster permission or not. You can refer...
- 1 kudos
- 2189 Views
- 1 replies
- 0 kudos
Databricks workspace creation using Terraform and storage configuration IAM Role Arn
Hi,When creating a new AWS Databricks workspace in account console manually, it appears that IAM Role ARN is mandatory parameter when configuring a Storage ConfigurationHowever in the Terraform databricks_mws_storage_configurations resource there is ...
- 2189 Views
- 1 replies
- 0 kudos
- 0 kudos
Investigated this further... Terraform code to create the workspace resource "databricks_mws_workspaces" "databricks_ws" { provider = databricks.mws account_id = var.databricks_account_id aws_region = var.region workspace_name = "databricks-w...
- 0 kudos
- 4663 Views
- 1 replies
- 0 kudos
Editing the SPN name in databricks
As a workspace administrator, I'm finding that I cannot change the Service Principal Name (SPN) immediately after registering it through its application ID in Databricks. It appears that Databricks has linked the SPN name with its application ID and ...
- 4663 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Priyam1 , Hope you are doing well! You can change the display name of the Service Principal using the account SCIM APIs : https://docs.databricks.com/api/azure/account/accountserviceprincipals/patch Please let me know if this helps and leave a l...
- 0 kudos
- 11822 Views
- 2 replies
- 1 kudos
Databricks AWS Secrets Manager access
I have a workspace deployed in AWS and need to read some secrets from AWS Secrets Manager in my notebook. I'm aware that there is no default process similar to Azure Key Vault, however I know that we can try to access it using boto3, but I'm stuck at...
- 11822 Views
- 2 replies
- 1 kudos
- 2015 Views
- 0 replies
- 0 kudos
unable to publish power bi dashboard with Databricks as source
Able to connect using personal access token and view the data in Hive; but when I publish to dashboard to share-point it results in :- PERMISSION_DENIED: You do not have permissions to autostart 1127-******-*********
- 2015 Views
- 0 replies
- 0 kudos
- 7013 Views
- 0 replies
- 0 kudos
Connect Azure DevOps pipeline to Private Link Databricks Workspace
Hello, i have been trying to get a pipeline in Azure DevOps to communicate with a Databricks Workspace that uses private link connection. I have tried setting up a service connection using a service principle that is also attached to the workspace, h...
- 7013 Views
- 0 replies
- 0 kudos
- 7677 Views
- 3 replies
- 3 kudos
Creating a private link for DBFS root storage
I am facing an issue with securing root DBFS storage access from Databricks.As I understand, Azure Databricks creates a default blob storage (a.k.a root storage) during the deployment process which is used for storing logs and telemetry. This storage...
- 7677 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @NadithK ,You need to create the private endpoint in the resource group where the workspace is deployed and not in the workspace-managed resource group. The workspace-managed resource group has the deny assignment which will not allow to create a ...
- 3 kudos
- 5778 Views
- 7 replies
- 0 kudos
Disable harmful commands at databricks cluster/workspace level
Hi,I want to prevent user from executing harmful commands like dbutils.fs.rm at cluster level/workspace level.Can we do this in databricks? Are there any classes/APIs that we can override?
- 5778 Views
- 7 replies
- 0 kudos
- 0 kudos
Hi ,I got the similar kind of problem too.I followed this put chmod -x /bin/rm in init.sh in workspace and add it at cluster your cluster level init script or global init script.this restricts the users to use rm command.Regards,Nikhil Srivastava@Pal...
- 0 kudos
- 3544 Views
- 3 replies
- 0 kudos
Service Principal can be deleted but permissions not managed
On Azure I added a service principal X to my databricks workspace. I therefore had the Service Prinicpal Manager role on that service principal X. I accidentally downgraded my rights to Service Principal User and now can's get my Managers role back. ...
- 3544 Views
- 3 replies
- 0 kudos
- 0 kudos
I believe so because the described effect was also observed in another databricks workspace, where I did NOT make the accidental change
- 0 kudos
