- 501 Views
- 2 replies
- 1 kudos
Vector Search index not indexing the whole Delta table
I have a Delta table that I’m trying to index but when I try to create a vector search index with either the UI or the Python SDK, it only indexes 1 row out of my 3000 rows. I have tried using different vector search endpoints. I have verified the fo...
- 501 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Nicolaus, Ensure that the Delta table is properly structured and that the columns used for indexing are correctly defined. Verify that the columns are of the correct data types and are not nullable.If the table is being populated through trickle...
- 1 kudos
- 1650 Views
- 2 replies
- 0 kudos
Displaying Dataframes with ipywidgets.Output is Adding Unexpected Commas
I am currently working in a databricks notebook and using an ipywidgets.Output to display a pandas Dataframe. Because spark.DataFrame cannot be displayed in an ipywidgets.Output widget, I have been using:import pandas as pd import numpy as np import ...
- 1650 Views
- 2 replies
- 0 kudos
- 408 Views
- 2 replies
- 0 kudos
Mounts in Databricks
How is it possible to prohibit a certain user from being able to see the mounts created in Databricks. Even if the user issues the %fs ls /mnt command, it doesn't return anything.
- 408 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Elcio , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedback n...
- 0 kudos
- 473 Views
- 1 replies
- 0 kudos
Asset Bundle template: ignore file based on variable value
Hello,I am creating an asset-bundle-template which should be able to include/exclude certain files, depending on the value of a parameter, e.g. if parameter "include_file" is set to true, then the bundle init command should include the template file(...
- 473 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @GeKo, Excluding Assets from Unity Builds Using AssetBundles Excluding Assets from the Main Game Build There are a few ways to exclude assets from the main game build in Unity while still using them in AssetBundles: 1. Organize Assets into Subfol...
- 0 kudos
- 1867 Views
- 6 replies
- 3 kudos
Driver and worker node utalisation
Hi all! Can anyone tell me if having a worker node(s) of the same type as my driver node make a difference performance-wise if I am running normal python code on a notebook as a job on this cluster? I am running mostly machine learning libraries such...
- 1867 Views
- 6 replies
- 3 kudos
- 997 Views
- 2 replies
- 1 kudos
Resolved! Profile setup for databricks Account
Hi Team,we usually do setup profile for databrikcs at workspace level, and i have done this using host url and token which is working fine like below.[DEFAULT]host = workspaceurlusername = ****password = tokenNow my question is how we set for databri...
- 997 Views
- 2 replies
- 1 kudos
- 1 kudos
You cannot create a PAT token for you Account Console authentication, on this case you will have 2 options to authenticate:Using Basic auth, on which on your profile you will set you username and password to log in, same as you do to log in in the br...
- 1 kudos
- 516 Views
- 1 replies
- 1 kudos
Databricks community group in Kerala
Calling All Data Enthusiasts in Kerala! Hey everyone,I'm excited about the idea of launching a Databricks Community Group here in Kerala! This group would be a hub for learning, sharing knowledge, and networking among data enthusiasts, analysts, a...
- 516 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @AswinGovindan77 , Do you have an estimate of how many enthusiasts will be joining you? If you provide a rough number, we can create a group for you on Community!
- 1 kudos
- 2132 Views
- 6 replies
- 2 kudos
Can't create branch of public git repo
Hi,I have cloned a public git repo into my Databricks account. It's a repo associated with an online training course. I'd like to work through the notebooks, maybe make some changes and updates, etc., but I'd also like to keep a clean copy of it. M...
- 2132 Views
- 6 replies
- 2 kudos
- 2 kudos
It occurs to me that one valid solution to this problem is simply to fork the repo and work there. Pretty standard approach, I guess, although not something I've ever been in the habit of doing.
- 2 kudos
- 464 Views
- 1 replies
- 1 kudos
Resolved! DATABRICKS FESTVAL VOUCHER
what should be the timezone which we have to consider for earning the voucher
- 464 Views
- 1 replies
- 1 kudos
- 1 kudos
@Aman_Dahiya Guessing you are referring to this event! There is no categorization for time zones, it is open to all. You can earn the voucher regardless of your timezone.
- 1 kudos
- 1237 Views
- 3 replies
- 0 kudos
How do Databricks handle the utilization of Azure Reserved Instances?
We are trying to use Azure Reserved Instances for Databricks clusters as we have some consistent workload.Does Databricks automatically select the reserved instances when they are available (assuming we keep the on-demand instance type in the cluster...
- 1237 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @NathanL, Does Databricks automatically select the reserved instances when they are available (assuming we keep the on-demand instance type in the cluster config)? Or is there explicit configuration we need to set to have Databricks use those re...
- 0 kudos
- 5091 Views
- 4 replies
- 0 kudos
Insufficient privileges:User does not have permission SELECT on any file
Hello,after switching to "shared cluster" usage a python job is failing with error message: Py4JJavaError: An error occurred while calling o877.load. : org.apache.spark.SparkSecurityException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User...
- 5091 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @sravs_227 ,the issue was, that the checkpoint directory (while reading from kafka) was set to a dbfs folder. We switched this now to also UC volume
- 0 kudos
- 464 Views
- 1 replies
- 1 kudos
Resolved! register for Databricks Community Edition
I've been trying to register for Databricks Community Edition for 3 days, to study the platform, but I'm experiencing an error when registering ("An error has occurred. Please try again later."). Could anyone help me, how do I register for the commun...
- 464 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @JoaoJr, Thank you for contacting Databricks Community Discussion Forum. Please note that for any issues related to the Databricks Community Edition product, you can find helpful resources here. If you encounter any difficulties beyond what's co...
- 1 kudos
- 1459 Views
- 3 replies
- 0 kudos
Feature Store UI lagging
The feature store UI is lagging (Azure Databricks). It is extremely slow to update - takes several days from creating a feature table before it appears in the UI. Anyone share the same experience?
- 1459 Views
- 3 replies
- 0 kudos
- 0 kudos
Anyone having the same experience here? The UI is still extremely slow to update - it takes days before the already registered feature tables appear in the UI. It makes the UI useless for browsing features! I really hope that Databricks is looking in...
- 0 kudos
- 901 Views
- 2 replies
- 1 kudos
Maintaining Order Consistency: Table Creation in Databricks SQL vs. DLT Pipeline
I have a CTE table with the below names as values. My objective is to create another table by concatenating all the rows from the CTE table in ascending order, resulting in the final output sequence: "Abi, Rahul, ram, Siva". When executing the query ...
- 901 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Archana_Mathan, did you solve this problem by any chance? I am having a similar problem maintaining the order of the dlt.Thank you,
- 1 kudos
- 885 Views
- 2 replies
- 2 kudos
Resolved! Pandas .to_excel not saving to workspace
Hi!I was always able to export pandas datasets to my workspace on databricks, just like in a local machine using df.to_excel('filename', index=False).but since i got some permissions updates to some new catalogs, when i try to export a dataset using ...
- 885 Views
- 2 replies
- 2 kudos
- 2 kudos
Just to update the situation, and to help anyone with the same problem in the future:What hapened was thas my cluster was updated to use Unity Catalogs, and with that i had to specify the full path to my workspace.df_pandas.to_excel('/Workspace/Users...
- 2 kudos
