- 962 Views
- 1 replies
- 0 kudos
How to NOT install or disable or uninstall Databricks Delta Live Tables dlt module on jobs cluster?
I need to NOT have the Databricks Delta Live Tables (DLT) Python stub installed on job cluster b/c of naming conflict w/ pip library dlt (and I also don't need delta live tables).There is no "simple" way of uninstalling. It's not installed via pip as...
- 962 Views
- 1 replies
- 0 kudos
- 0 kudos
For anyone facing a similar problem: I've addressed the issue of the dlt module conflict on my job cluster, by using an init script to remove the dlt module from the cluster's Python environment.Simply by doing:%bash #!/bin/bash rm -rf /databricks/sp...
- 0 kudos
- 1139 Views
- 1 replies
- 0 kudos
legacy repo error fetching git status files over 200MB
Working directory contains files that exceed the allowed limit of 200 MB. how to solve this?
- 1139 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @JessieWen ,What you can do besides removing some files from the repo, is to use "Sparce mode" and select only certain paths to be synchronized with Databricks repos. Hope it helps
- 0 kudos
- 7063 Views
- 8 replies
- 0 kudos
Databricks SQL connectivity in Python with Service Principals
Tried to use M2M OAuth connectivity on Databricks SQL Warehouse in Python:from databricks.sdk.core import Config, oauth_service_principal from databricks import sql .... config = Config(host=f"https://{host}", client_...
- 7063 Views
- 8 replies
- 0 kudos
- 0 kudos
Did anyone get this to work? I have tried the code above but I get a slightly different error but I don't see the same level of details from the logs2024-10-04 14:59:25,508 [databricks.sdk][DEBUG] Attempting to configure auth: pat2024-10-04 14:59:25,...
- 0 kudos
- 13285 Views
- 1 replies
- 0 kudos
Connect Community Edition to Power BI Desktop
I have submitted this question several times to Databricks over the past few weeks, and I have gotten no response at all, not even an acknowledgement that my request was received.Please help.How can I connect a certain dataset in Databricks Community...
- 13285 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Retired_mod,It seams the Commnity Edition doesn't let us to generate the personal-access-token any more. Could you let us know some where we can get the token in the Comminity Edition?Thanks.
- 0 kudos
- 914 Views
- 1 replies
- 0 kudos
cluster administrator
Is individual cluster more cost effective or shared group cluster?
- 914 Views
- 1 replies
- 0 kudos
- 0 kudos
This is very generic, it depends upon use case. If you have a bunch of users trying to read data from catalogs, and perform data analysis or analytics creating a common cluster will be more cost effective and provided better performance. Also, largel...
- 0 kudos
- 1778 Views
- 1 replies
- 0 kudos
How to assign user group for email notification in databricks Alerts
How can I assign a azure databricks user group to an alert for notification?Current scenario is whenever we need to add a user for alert email notification we are manually adding that user email address to each we setup (more than 100) which is very ...
- 1778 Views
- 1 replies
- 0 kudos
- 0 kudos
One option is to handle the logic inside the python notebook to trigger alerts using emali and smtp lib which accepts databricks local groups and AD groups that are synched.
- 0 kudos
- 1990 Views
- 1 replies
- 0 kudos
Resolved! Resource organization in a large company
Hello,We are using Azure Databricks in a single tenant. We will have many teams working in multiple (Unity Enabled) Workspaces using a variety of Catalogs, External Locations, Storage Credentials, ect. Some of those resources will be shared (e.g., an...
- 1990 Views
- 1 replies
- 0 kudos
- 0 kudos
We are using Azure Databricks in a single tenant. We will have many teams working in multiple (Unity Enabled) Workspaces using a variety of Catalogs, External Locations, Storage Credentials, etc. Some of those resources will be shared (e.g., an Exter...
- 0 kudos
- 2852 Views
- 3 replies
- 0 kudos
HTTPSConnectionPool(host='sandvik.peakon.com', port=443): Max retries exceeded with url: /api/v1/seg
while connecting to an api from databricks notebook with the bearer token I am getting the below errorHTTPSConnectionPool(host='sandvik.peakon.com', port=443): Max retries exceeded with url: /api/v1/segments?page=1 (Caused by SSLError(SSLCertVerifica...
- 2852 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @SunilSamal The error you are encountering, SSLCertVerificationError, indicates that the SSL certificate verification failed because the local issuer certificate could not be obtained. This is a common issue when the SSL certificate chain is inco...
- 0 kudos
- 1999 Views
- 4 replies
- 1 kudos
Resolved! Unable to get S3 connection working
I can't get past the error below. I've read and reread the instructions several times at the URL below and for the life of me cannot figure out what I'm missing in my AWS setup. Any tips on how to track down my issue? https://docs.databricks.com/en/c...
- 1999 Views
- 4 replies
- 1 kudos
- 1 kudos
I got it working, there was a weird typo where the role ARN was duplicated. Thanks.
- 1 kudos
- 958 Views
- 1 replies
- 0 kudos
samples catalog doesnt have an information schema
we are looking to do an integration with databricks, and i've noticed that the samples database doesn't have an INFORMATION_SCHEMA we rely on the existence of the information_schema to help us understand what views / tables exist in each catalog. w...
- 958 Views
- 1 replies
- 0 kudos
- 0 kudos
The "samples" catalog in Databricks does not have an INFORMATION_SCHEMA because it is designed primarily for demonstration and educational purposes, rather than for production use. This schema is typically included in catalogs created on Unity Catalo...
- 0 kudos
- 1987 Views
- 1 replies
- 0 kudos
Issue when connecting to Databricks using OAuth 2.0 token connection string
Following the instructions on Authentication settings for the Databricks ODBC Driver | Databricks on AWS I have created both Personal access token and OAuth 2.0 token connection options in an application. However, I realized that, when I use OAuth 2....
- 1987 Views
- 1 replies
- 0 kudos
- 0 kudos
This will require further analysis, based on the information it seems there is no direct way to restrict this. Are you able to submit a support case so this can be analyzed?
- 0 kudos
- 2760 Views
- 1 replies
- 0 kudos
Resolved! Databricks connect - SQL Server - Login error with all purpose cluster
Hello everyone,I'm using the Databricks connect feature to connect to an SQL server on cloud.I created a foreign catalog based on the connection but whenever I try to access the tables, I get a login error : I have tried with a serverless cluster and...
- 2760 Views
- 1 replies
- 0 kudos
- 0 kudos
Solved.Turns out it was a networking issue, once the subnets from databricks where allowed by the cloud sql server we managed to connect. The error message is misleading because the credentials were correct
- 0 kudos
- 6351 Views
- 3 replies
- 1 kudos
Resolved! PERMISSION_DENIED: User is not an owner of Table/Schema
Hi,We have recently added a service principal for running and managing all of our jobs. The service principal has ALL PRIVILEGES to our catalogs/schemas/and table. But we're still seeing the error message `PERMISSION_DENIED: User is not an owner of T...
- 6351 Views
- 3 replies
- 1 kudos
- 1 kudos
I think the feedback button is the right place. At least I don't know of another way.
- 1 kudos
- 9477 Views
- 1 replies
- 0 kudos
Resolved! [DATA_SOURCE_NOT_FOUND] Failed to find data source
Context:Hello, I was using a workflow for a periodic process, with my team we were using a Job Compute, but the libraries were not working (even though we had a PIP_EXTRA_INDEX_URL defined in the Environment Variables of the Cluster, so we now use a ...
- 9477 Views
- 1 replies
- 0 kudos
- 0 kudos
I installed in the cluster this library:spark_mssql_connector_2_12_1_4_0_BETA.jarA colleague passed me this .jar file. It seems that can be obtained from here: https://github.com/microsoft/sql-spark-connector/releases.This allows the task to end succ...
- 0 kudos
- 1732 Views
- 1 replies
- 0 kudos
Exam for Databricks Certified Data Engineer Associte
My Databricks professional data Engineer certification exam got suspended. My Exam just went for half hour, it was showing me error for eye movement when I was reading question, exam suspended on 11th of July 2024 and still showing in progress assess...
- 1732 Views
- 1 replies
- 0 kudos
- 0 kudos
I'm sorry to hear your exam was suspended. Please file a ticket with our support team and allow the support team 24-48 hours for a resolution. You should also review this documentation:Room requirementsBehavioral considerations
- 0 kudos
