I have a Delta table that I’m trying to index but when I try to create a vector search index with either the UI or the Python SDK, it only indexes 1 row out of my 3000 rows. I have tried using different vector search endpoints. I have verified the fo...
I am currently working in a databricks notebook and using an ipywidgets.Output to display a pandas Dataframe. Because spark.DataFrame cannot be displayed in an ipywidgets.Output widget, I have been using:import pandas as pd import numpy as np import ...
I have this issue also, and the listed steps do not resolve it.
How is it possible to prohibit a certain user from being able to see the mounts created in Databricks. Even if the user issues the %fs ls /mnt command, it doesn't return anything.
Hi @Elcio , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedback n...
Hi all! Can anyone tell me if having a worker node(s) of the same type as my driver node make a difference performance-wise if I am running normal python code on a notebook as a job on this cluster? I am running mostly machine learning libraries such...
Hi Team,we usually do setup profile for databrikcs at workspace level, and i have done this using host url and token which is working fine like below.[DEFAULT]host = workspaceurlusername = ****password = tokenNow my question is how we set for databri...
You cannot create a PAT token for you Account Console authentication, on this case you will have 2 options to authenticate:Using Basic auth, on which on your profile you will set you username and password to log in, same as you do to log in in the br...
Hello Team,I am new to DAB and running it for the first time through the Databricks CLI.The bundle validation is successful but while running it errors out error="expected a KEY of the resource to run".Can anyone help me on what to check to resolve t...
Calling All Data Enthusiasts in Kerala! Hey everyone,I'm excited about the idea of launching a Databricks Community Group here in Kerala! This group would be a hub for learning, sharing knowledge, and networking among data enthusiasts, analysts, a...
Hi @AswinGovindan77 , Do you have an estimate of how many enthusiasts will be joining you? If you provide a rough number, we can create a group for you on Community!
Hi,I have cloned a public git repo into my Databricks account. It's a repo associated with an online training course. I'd like to work through the notebooks, maybe make some changes and updates, etc., but I'd also like to keep a clean copy of it. M...
It occurs to me that one valid solution to this problem is simply to fork the repo and work there. Pretty standard approach, I guess, although not something I've ever been in the habit of doing.
what should be the timezone which we have to consider for earning the voucher
@Aman_Dahiya Guessing you are referring to this event! There is no categorization for time zones, it is open to all. You can earn the voucher regardless of your timezone.
Hello,I am creating an asset-bundle-template which should be able to include/exclude certain files, depending on the value of a parameter, e.g. if parameter "include_file" is set to true, then the bundle init command should include the template file(...
Hello,after switching to "shared cluster" usage a python job is failing with error message: Py4JJavaError: An error occurred while calling o877.load. : org.apache.spark.SparkSecurityException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User...
Hi @sravs_227 ,the issue was, that the checkpoint directory (while reading from kafka) was set to a dbfs folder. We switched this now to also UC volume
Hello everyone,In a workflow, is it possible to trigger a retry only for a specific error on a single task? I want the workflow UI to show a run as failed for both managed and unmanaged errors, but I don't want to trigger the retry for managed error...
The feature store UI is lagging (Azure Databricks). It is extremely slow to update - takes several days from creating a feature table before it appears in the UI. Anyone share the same experience?
Anyone having the same experience here? The UI is still extremely slow to update - it takes days before the already registered feature tables appear in the UI. It makes the UI useless for browsing features! I really hope that Databricks is looking in...
We are trying to use Azure Reserved Instances for Databricks clusters as we have some consistent workload.Does Databricks automatically select the reserved instances when they are available (assuming we keep the on-demand instance type in the cluster...
I have a CTE table with the below names as values. My objective is to create another table by concatenating all the rows from the CTE table in ascending order, resulting in the final output sequence: "Abi, Rahul, ram, Siva". When executing the query ...
Hi @Archana_Mathan, did you solve this problem by any chance? I am having a similar problem maintaining the order of the dlt.Thank you,
