- 1246 Views
- 1 replies
- 0 kudos
Using model serving from databricks privacy issue
When using Databrick's model serving to query Llama3, I noticed the endpoint URL is my databricks instance. Does that still mean data is sent to databricks process by databricks? If so, does databricks keep/use any of the data sent to model serving ...
- 1246 Views
- 1 replies
- 0 kudos
- 0 kudos
Related question, when databricks process requests for foundational model, I noticed the latency is pretty small, wondering what kind of processing power is used on databricks side? I am interested in hosting model ourselves so wondering what type of...
- 0 kudos
- 2996 Views
- 1 replies
- 1 kudos
Vector Search index not indexing the whole Delta table
I have a Delta table that I’m trying to index but when I try to create a vector search index with either the UI or the Python SDK, it only indexes 1 row out of my 3000 rows. I have tried using different vector search endpoints. I have verified the fo...
- 2996 Views
- 1 replies
- 1 kudos
- 3715 Views
- 1 replies
- 0 kudos
Displaying Dataframes with ipywidgets.Output is Adding Unexpected Commas
I am currently working in a databricks notebook and using an ipywidgets.Output to display a pandas Dataframe. Because spark.DataFrame cannot be displayed in an ipywidgets.Output widget, I have been using:import pandas as pd import numpy as np import ...
- 3715 Views
- 1 replies
- 0 kudos
- 0 kudos
I have this issue also, and the listed steps do not resolve it.
- 0 kudos
- 7329 Views
- 7 replies
- 3 kudos
Writing a single huge dataframe into Azure SQL Database using JDBC
Hi All,I am currently trying to read data from a materialized view as a single dataframe which contains around 10M of rows and then write it into an Azure SQL database. However, I don't see the spark job moving a bit even an hour is passed. I have al...
- 7329 Views
- 7 replies
- 3 kudos
- 3 kudos
Hi @yeungcase , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedba...
- 3 kudos
- 1289 Views
- 1 replies
- 0 kudos
Mounts in Databricks
How is it possible to prohibit a certain user from being able to see the mounts created in Databricks. Even if the user issues the %fs ls /mnt command, it doesn't return anything.
- 1289 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Elcio , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedback n...
- 0 kudos
- 8499 Views
- 6 replies
- 3 kudos
Driver and worker node utalisation
Hi all! Can anyone tell me if having a worker node(s) of the same type as my driver node make a difference performance-wise if I am running normal python code on a notebook as a job on this cluster? I am running mostly machine learning libraries such...
- 8499 Views
- 6 replies
- 3 kudos
- 3686 Views
- 2 replies
- 1 kudos
Resolved! Profile setup for databricks Account
Hi Team,we usually do setup profile for databrikcs at workspace level, and i have done this using host url and token which is working fine like below.[DEFAULT]host = workspaceurlusername = ****password = tokenNow my question is how we set for databri...
- 3686 Views
- 2 replies
- 1 kudos
- 1 kudos
You cannot create a PAT token for you Account Console authentication, on this case you will have 2 options to authenticate:Using Basic auth, on which on your profile you will set you username and password to log in, same as you do to log in in the br...
- 1 kudos
- 1342 Views
- 0 replies
- 0 kudos
Databricks Asset Bundle Error 'KEY of the resource to run"
Hello Team,I am new to DAB and running it for the first time through the Databricks CLI.The bundle validation is successful but while running it errors out error="expected a KEY of the resource to run".Can anyone help me on what to check to resolve t...
- 1342 Views
- 0 replies
- 0 kudos
- 7808 Views
- 6 replies
- 2 kudos
Can't create branch of public git repo
Hi,I have cloned a public git repo into my Databricks account. It's a repo associated with an online training course. I'd like to work through the notebooks, maybe make some changes and updates, etc., but I'd also like to keep a clean copy of it. M...
- 7808 Views
- 6 replies
- 2 kudos
- 2 kudos
It occurs to me that one valid solution to this problem is simply to fork the repo and work there. Pretty standard approach, I guess, although not something I've ever been in the habit of doing.
- 2 kudos
- 1364 Views
- 1 replies
- 1 kudos
Resolved! DATABRICKS FESTVAL VOUCHER
what should be the timezone which we have to consider for earning the voucher
- 1364 Views
- 1 replies
- 1 kudos
- 1 kudos
@Aman_Dahiya Guessing you are referring to this event! There is no categorization for time zones, it is open to all. You can earn the voucher regardless of your timezone.
- 1 kudos
- 2991 Views
- 3 replies
- 2 kudos
Resolved! When does the cost of JOB COMPUTE start to be calculated?
I'm trying to run a workflow with job compute.Job compute needs to be pending for about 5 to 7 minutes before executing the workflow. I think it takes time to find a suitable instance in the cloud, configure the environment, install libraries, etc.An...
- 2991 Views
- 3 replies
- 2 kudos
- 2 kudos
There is a big discrepancy to take note of here. It is the difference between Databricks job costs (DBUs) and billing for the cloud provider (the actual VM doing the the work). Billing for the Databricks job starts when the Spark context is being ini...
- 2 kudos
- 1507 Views
- 0 replies
- 0 kudos
Asset Bundle template: ignore file based on variable value
Hello,I am creating an asset-bundle-template which should be able to include/exclude certain files, depending on the value of a parameter, e.g. if parameter "include_file" is set to true, then the bundle init command should include the template file(...
- 1507 Views
- 0 replies
- 0 kudos
- 1594 Views
- 1 replies
- 1 kudos
Error with: %run ../Includes/Classroom-Setup-SQL
Hi Guys,Just started the ASP 2.1L Spare SQL Lab and I get this error, when I run the first setup SQL command:%run ../Includes/Classroom-Setup-SQLThe execution of this command did not finish successfullyPython interpreter will be restarted.Python inte...
- 1594 Views
- 1 replies
- 1 kudos
- 1 kudos
You can create a cluster compatible with the notebooks. So It will work.
- 1 kudos
- 2471 Views
- 1 replies
- 1 kudos
Databricks CLI bundle run multiple jobs
Hi!I am using bundles to deploy various workflows, and as part of a CI pipeline, I want to run integration tests. They are in notebooks that I deploy and run with databricks CLI bundle with Azure DevOps.This allows me to run only one job at a time as...
- 2471 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Retired_mod and thanks for your reply.I see, so all tests will need to be defined in one config file under one job.
- 1 kudos
- 65894 Views
- 3 replies
- 3 kudos
Resolved! How to import excel on databricks
To import an Excel file into Databricks, you can follow these general steps:1. **Upload the Excel File**:- Go to the Databricks workspace or cluster where you want to work.- Navigate to the location where you want to upload the Excel file.- Click on ...
- 65894 Views
- 3 replies
- 3 kudos
- 3 kudos
The question here is how to read the multi-excel files based on path.The mentioned solution interacts with one file only, do we have the ability to read all the Excel files in the folder?
- 3 kudos
