Databricks Community is an open-source platform for data enthusiasts and professionals to discuss, share insights, and collaborate on everything related to Databricks. Members can ask questions, share knowledge, and support each other in an environment that ensures respectful interactions.
I'm looking for a solution where 1. Need a delta table that saves the pointers(path) of the images in volume 2. Train a Pytorch, tensorflow or keras model with the data from delta lake. I tried mu...
I am trying to execute pandas UDF in databricks. It gives me the following error on serverless compute, File /local_disk0/.ephemeral_nfs/envs/pythonEnv-b11ff17c-9b25-4ccb-927d-06a7d1ca7221/lib/pyt...
We use a "warmup" mechanism to get our DBR instance pool into a state where it has at-least-N instances. The logic is: For N repetitions: Request a new DBR cluster in the pool (which causes the po...
Hello, I'm currently using Databricks on AWS for some basic ETL where the resulting data is stored as Hive external delta tables. Even though Unity catalog is disabled, table access control is ...
...RIGIN_COUNTRY_NAME, count. The dataset has a total of 256 rows. Here’s the code I'm running: data = (spark
.read
.format('csv')
.option('inferSchema', 'true')
.option('header', 'true...
Hi, I have a py notebook that I want to execute in an automated manner. One way I found this was to attach this to a job/task and hit it using the api from my local. However this seems to be adding s...
Hello, hope you are doing good. When on the search page, it seems it's not searching for code inside .py files but rather only the filename. Is there an option somewhere I'm missing to be able to...
Hello, hope you are doing well. We are facing an issue when running .py files. This is fairly recent and we were not experiencing this issue last week. As shown in the screenshots below, the .py ...
I have been actively using the Databricks Partner Academy for the past three years through my current organization. As I am planning to transition to a new company, I would like to ensure continued a...