- 518 Views
- 2 replies
- 0 kudos
Azure Databricks Serverless Compute
Hello,Looking for documents related to Azure Databricks Serverless Compute. What are the things we need to consider for security point of view when we decide to use serverless compute?
- 518 Views
- 2 replies
- 0 kudos
- 0 kudos
These steps are really helpful. I especially appreciate the reminder to check my credentials and consider browser-related issues, as those are often overlooked. I'll make sure to clear my cache and cookies first, and if that doesn't work, I’ll try us...
- 0 kudos
- 385 Views
- 1 replies
- 0 kudos
How to NOT install or disable or uninstall Databricks Delta Live Tables dlt module on jobs cluster?
I need to NOT have the Databricks Delta Live Tables (DLT) Python stub installed on job cluster b/c of naming conflict w/ pip library dlt (and I also don't need delta live tables).There is no "simple" way of uninstalling. It's not installed via pip as...
- 385 Views
- 1 replies
- 0 kudos
- 0 kudos
For anyone facing a similar problem: I've addressed the issue of the dlt module conflict on my job cluster, by using an init script to remove the dlt module from the cluster's Python environment.Simply by doing:%bash #!/bin/bash rm -rf /databricks/sp...
- 0 kudos
- 352 Views
- 0 replies
- 0 kudos
Ray cannot detect GPU on the cluster
I am trying to run ray on databricks for chunking and embedding tasks. The cluster I’m using is:g4dn.xlarge1-4 workers with 4-16 cores1 GPU and 16GB memoryI have set spark.task.resource.gpu.amount to 0.5 currently.This is how I have setup my ray clus...
- 352 Views
- 0 replies
- 0 kudos
- 422 Views
- 1 replies
- 0 kudos
legacy repo error fetching git status files over 200MB
Working directory contains files that exceed the allowed limit of 200 MB. how to solve this?
- 422 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @JessieWen ,What you can do besides removing some files from the repo, is to use "Sparce mode" and select only certain paths to be synchronized with Databricks repos. Hope it helps
- 0 kudos
- 956 Views
- 0 replies
- 0 kudos
Databricks - Cost difference between Job Clusters and DLT
Wanted to know about the cost comparison and certain specific feature details between job clusters and DLT,Per the Pricing page (based on both Azure Pricing and Databricks Pricing page) following is the understanding - Region: US EastProvisionedJobs ...
- 956 Views
- 0 replies
- 0 kudos
- 4591 Views
- 8 replies
- 0 kudos
Databricks SQL connectivity in Python with Service Principals
Tried to use M2M OAuth connectivity on Databricks SQL Warehouse in Python:from databricks.sdk.core import Config, oauth_service_principal from databricks import sql .... config = Config(host=f"https://{host}", client_...
- 4591 Views
- 8 replies
- 0 kudos
- 0 kudos
Did anyone get this to work? I have tried the code above but I get a slightly different error but I don't see the same level of details from the logs2024-10-04 14:59:25,508 [databricks.sdk][DEBUG] Attempting to configure auth: pat2024-10-04 14:59:25,...
- 0 kudos
- 12415 Views
- 1 replies
- 0 kudos
Connect Community Edition to Power BI Desktop
I have submitted this question several times to Databricks over the past few weeks, and I have gotten no response at all, not even an acknowledgement that my request was received.Please help.How can I connect a certain dataset in Databricks Community...
- 12415 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Retired_mod,It seams the Commnity Edition doesn't let us to generate the personal-access-token any more. Could you let us know some where we can get the token in the Comminity Edition?Thanks.
- 0 kudos
- 531 Views
- 1 replies
- 0 kudos
cluster administrator
Is individual cluster more cost effective or shared group cluster?
- 531 Views
- 1 replies
- 0 kudos
- 0 kudos
This is very generic, it depends upon use case. If you have a bunch of users trying to read data from catalogs, and perform data analysis or analytics creating a common cluster will be more cost effective and provided better performance. Also, largel...
- 0 kudos
- 769 Views
- 1 replies
- 0 kudos
How to assign user group for email notification in databricks Alerts
How can I assign a azure databricks user group to an alert for notification?Current scenario is whenever we need to add a user for alert email notification we are manually adding that user email address to each we setup (more than 100) which is very ...
- 769 Views
- 1 replies
- 0 kudos
- 0 kudos
One option is to handle the logic inside the python notebook to trigger alerts using emali and smtp lib which accepts databricks local groups and AD groups that are synched.
- 0 kudos
- 383 Views
- 0 replies
- 0 kudos
Creating Group in Terraform using external_id
The documentation here doesn't give much information about how to use `external_id` when creating a new group. If I reference the object_id for an Azure AD Group, the databricks group gets created but the members from the AD group are not added, nor ...
- 383 Views
- 0 replies
- 0 kudos
- 803 Views
- 1 replies
- 0 kudos
Resolved! Resource organization in a large company
Hello,We are using Azure Databricks in a single tenant. We will have many teams working in multiple (Unity Enabled) Workspaces using a variety of Catalogs, External Locations, Storage Credentials, ect. Some of those resources will be shared (e.g., an...
- 803 Views
- 1 replies
- 0 kudos
- 0 kudos
We are using Azure Databricks in a single tenant. We will have many teams working in multiple (Unity Enabled) Workspaces using a variety of Catalogs, External Locations, Storage Credentials, etc. Some of those resources will be shared (e.g., an Exter...
- 0 kudos
- 976 Views
- 3 replies
- 0 kudos
HTTPSConnectionPool(host='sandvik.peakon.com', port=443): Max retries exceeded with url: /api/v1/seg
while connecting to an api from databricks notebook with the bearer token I am getting the below errorHTTPSConnectionPool(host='sandvik.peakon.com', port=443): Max retries exceeded with url: /api/v1/segments?page=1 (Caused by SSLError(SSLCertVerifica...
- 976 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @SunilSamal The error you are encountering, SSLCertVerificationError, indicates that the SSL certificate verification failed because the local issuer certificate could not be obtained. This is a common issue when the SSL certificate chain is inco...
- 0 kudos
- 679 Views
- 0 replies
- 0 kudos
Unity Catalog Volume mounting broken by cluster environment variables (http proxy)
Hello all,I have a slightly niche issue here, albeit one that others are likely to run into.Using databricks on Azure, my organisation has included extended our WAN into the cloud, so that all compute clusters are granted a private IP address that ca...
- 679 Views
- 0 replies
- 0 kudos
- 195 Views
- 0 replies
- 0 kudos
Delete Users that are Maintenance Readers
I am an Account Admin at Databricks (Azure), and trying to delete users that are being offboarded.I have managed to delete most users. However, for a couple, I get the following message (see screenshot):ABORTED: Account <account> is read-only during ...
- 195 Views
- 0 replies
- 0 kudos
- 348 Views
- 0 replies
- 0 kudos
VS Code Databricks Connect Cluster Configuration
I am currently setting up the VSCode extension for Databricks Connect, and it’s working fine so far. However, I have a question about cluster configurations. I want to access Unity Catalog from VSCode through the extension, and I’ve noticed that I ca...
- 348 Views
- 0 replies
- 0 kudos
