Databricks Community

Best Database for facial recognition/ Fast comparisons of Euclidean distance

Hello people,I'm trying to build a facial recognition application, and I have a working API, that takes in an image of a face and spits out a vector that encodes it. I need to run this on a million faces, store them in a db and when the system goes o...

Databricks delta lake

Can we use databricks delta lake as a data warehouse kind of thing where business analysts can explore data according to their needs ? Delta lake provides following features which I think supports this idea support to sql syntaxprovide ACID guarante...

On Azure Databricks, is it true that a customer cannot change the VNET CIDR range for a workspace and the only way to do so is to deploy a new workspace? If so, why?

Is possible to use jupyter extensions in databricks?

Hi, we need create an interactive map from ipyleaflet library and this use jupyterlab extensionjupyter labextension install @jupyter-widgets/jupyterlab-manager jupyter-leafletWe achieved to show with displayHTML but we lose the widget events

Azure Databricks cluster external IP ranges to be locked down for firewalls

The Spark driver / executors have callbacks to an external service. I need to find out the IP ranges to be locked down for the firewall on the external service. Here is what I found from Azure (https://www.microsoft.com/en-us/download/details.aspx?...

Cluster Pending

Hello, Relatively new to Databricks and I've been using the Community Edition for a little bit now. I've recently been having more and more issues with my clusters pending until they time out before ever starting up. I've seen a few other posts here...

Test Latesst Question1

Test Latesst Question

New task orchestration tool

Hi everybody! Is it possible to use one job cluster for multiple tasks using new task orchestration tool? If so then please tell how. If it's impossible then this new tool looks useless Regards, Daniil.

Where is dbfs mounted with community edition?

The the regular version of databricks, the dbfs is mounted at /dbfs. This does not seem to be the case with community edition. I am seeking more details.

How to run databricks notebooks on databricks cluster from VSCODE?

I want to use the VS Code IDE to edit/write/work on jupyter notebooks but have the code run off of the databricks cluster. Does anyone know of a way for me to force the jupyter notebook server on VS Code to point towards the databricks cluster and e...

Structured streaming from an overwrite delta path

Hi experts, I need to ingest data from an existing delta path to my own delta lake. The dataflow is as shown in the diagram: Data team reads full snapshot of a database table and overwrite to a delta path. This is done many times per day, but...

GroupBy in a multi node environment

I have a group of rows with Information on a nested product calls. example- Trxn1-product1-caller1-local1 Trxn1-Product1-local1-local2 Trxn1-Product1-local2-local3 here’s is a expected calls for a product product1-caller1-local1 Product1-local1-loc...

What is the equivalent command for constructing the filepath in Databricks on AWS? filepath = f"{working_dir}/keras_checkpoint_weights.ckpt"

dbutils.fs.mkdirs("/foobar/")See https://docs.databricks.com/data/databricks-file-system.html

Databricks-connect: is it safe to store my PAT in plaintext?

i am getting began with databricks-connect to hook up with my azure databricks cluster the use of a private get admission to token. it looks 192.168.1.254 like this token is being stored in a uncooked text report. if it is proper, is there a extra r...