cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Anonymous
by Not applicable
  • 4159 Views
  • 4 replies
  • 2 kudos

  Hello Everyone, I am thrilled to announce that we have our 6th winner for the raffle contest - @Yogita Chavan​ . Please join me in congratulating he...

 Hello Everyone,I am thrilled to announce that we have our 6th winner for the raffle contest - @Yogita Chavan​ . Please join me in congratulating her on this remarkable achievement!Your dedication and hard work have paid off, and we are delighted to ...

6
  • 4159 Views
  • 4 replies
  • 2 kudos
Latest Reply
yogu
Databricks Partner
  • 2 kudos

@Suteja Kanuri​ i haven't received swag until now as per our last mail i provided size after that I didn't receive any mail for tracking details

  • 2 kudos
3 More Replies
bshirdi
by New Contributor II
  • 10784 Views
  • 1 replies
  • 2 kudos

Getting HTTP 502 bad gateway error!

Hello all,I am suddenly getting an HTTP 502 and DRIVER_LIBRARY_INSTALLATION_FAILURE error during the Python library installation when the cluster gets initialized. I have around 10 Python packages out of which 2-3, packages always failed to install a...

image.png
  • 10784 Views
  • 1 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Bhargav Shir​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 2 kudos
Priyag1
by Honored Contributor II
  • 1993 Views
  • 2 replies
  • 10 kudos

How to improve visualization and dasboard concepts?

How to improve visualization and dasboard concepts?

  • 1993 Views
  • 2 replies
  • 10 kudos
Latest Reply
Anonymous
Not applicable
  • 10 kudos

Hi @Priyadarshini G​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 10 kudos
1 More Replies
carlosjrestr
by New Contributor III
  • 5076 Views
  • 1 replies
  • 1 kudos

Does Unity Catalog on Azure require premium blob storage tier?

from the docs I read : Create a storage container where the metastore’s managed table data will be stored.This storage container must be in a Premium performance Azure Data Lake Storage Gen2 account in the same region as the workspaces you want to us...

  • 5076 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Carlos Restrepo​ We haven't heard from you since the last response from @Kaniz Fatma​ â€‹, and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to o...

  • 1 kudos
pranavyadavbugy
by New Contributor II
  • 4256 Views
  • 1 replies
  • 1 kudos

Regarding Discount on certifications for students

Hi team,I'm a student is there any student discounts for students on certification if yes please let me know.Thanks

  • 4256 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Pranav Yadav​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 1 kudos
jch
by New Contributor III
  • 3058 Views
  • 1 replies
  • 0 kudos

Can Kaggle Kernels be used in Databricks notebooks?

suggestions).This one looks perfect kaggle kernels output rsrishav/starter-youtube-trending-video-dataset -p /path/to/dest but I'm not using CLI, I'm using a databricks notebook.I tried using this code but it doesn't work. data_path = 'rsrishav/youtu...

  • 3058 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @jch​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 0 kudos
Phani1
by Databricks MVP
  • 1639 Views
  • 1 replies
  • 0 kudos

Databricks Cross cloud (Databricks on AWS and Storage is on Azure)

We have a Databricks account on AWS and storage is on Azure Cloud (ADLS), because of cross-platform do we face any security and performance-related issues also is this feasible and supported? could you please provide your suggestions on this?

  • 1639 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Janga Reddy​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 0 kudos
guille-ci
by New Contributor
  • 1354 Views
  • 1 replies
  • 0 kudos

[bug] databricks jobs list not desplaying 2.0 created jobs

Hi! When I use `databricks jobs list --version=2.0` I get all jobs deployed using 2.0 and 2.1 API, however, when I use `databricks jobs list --version=2.1` I only get jobs deployed using 2.1 API. This is a behaviour that we've only experienced recent...

  • 1354 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Guillermo Sanchez​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 0 kudos
marksachin_k
by New Contributor
  • 3785 Views
  • 1 replies
  • 0 kudos

Python custom Logging on Databricks

I am planning to introduce a custom logging to the databricks workload. To achieve this I am using a python logging module. I am storing logs in driver memory "file:/tmp/" directory before I move those logs to blob storage. In my personal databricks ...

  • 3785 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @MARKSACHIN K​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 0 kudos
abhilashmhaisne
by New Contributor II
  • 2475 Views
  • 1 replies
  • 1 kudos

Error while creating workspace with 'Quickstart' on AWS

Cloudformation runs into error during 'assign Metastore' step.Following is the error from Cloudwatch:HTTP content: b'{ "error_code": "PERMISSION_DENIED", "message": "Cannot assign metastore to STANDARD tier workspace xyz" }What is the solution for th...

  • 2475 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Abhilash Mhaisne​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 1 kudos
rdobbss
by New Contributor II
  • 6733 Views
  • 3 replies
  • 3 kudos

How to use foreachbatch in deltalivetable or DLT?

I need to process some transformation on incoming data as a batch and want to know if there is way to use foreachbatch option in deltalivetable. I am using autoloader to load json files and then I need to apply foreachbatch and store results into ano...

  • 6733 Views
  • 3 replies
  • 3 kudos
Latest Reply
TomRenish
New Contributor III
  • 3 kudos

Not sure if this will apply to you or not...I was looking at the foreachbatch tool to reduce the workload of getting distinct data from a history table of 20million + records because the df.dropDuplicates() function was intermittently running out of ...

  • 3 kudos
2 More Replies
Oliver_Angelil
by Valued Contributor II
  • 11729 Views
  • 3 replies
  • 3 kudos

New SQL Dashboard feature never loads

I've created the sample Taxi dashboard, but the charts never load?

Screenshot 2023-05-11 at 13.39.59
  • 11729 Views
  • 3 replies
  • 3 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 3 kudos

Do you have the full error stack trace? does this happens frequently? whats the size of your cluster?

  • 3 kudos
2 More Replies
Prabhakar1
by New Contributor III
  • 19866 Views
  • 5 replies
  • 8 kudos

How Selenium Webdriver works on Azure Databricks? I am unable to run a simple code.

from selenium import webdriverfrom webdriver_manager.chrome import ChromeDriverManagerfrom selenium.webdriver.common.by import Byfrom selenium.webdriver.chrome.options import Optionsdrivers = webdriver.Chrome(ChromeDriverManager().install())drivers.g...

  • 19866 Views
  • 5 replies
  • 8 kudos
Latest Reply
Evan_MCK
Contributor
  • 8 kudos

I also got that error. What worked for me was downloading the chrome driver and ensuring its the latest version with shell scripts in the same notebook I used for web scraping. I could not use the web driver manager. You can see all the details here...

  • 8 kudos
4 More Replies
AJ270990
by Contributor III
  • 17148 Views
  • 4 replies
  • 6 kudos

Resolved! ChromeDriver installation in Databricks

I am working on a Webscraping logic and need to install Chrome driver. How can I install it in the Databricks workbook ?

  • 17148 Views
  • 4 replies
  • 6 kudos
Latest Reply
Evan_MCK
Contributor
  • 6 kudos

What worked for me was downloading the chrome driver and ensuring its the latest version with shell scripts in the same notebook I used for web scraping. You can see all the details here: https://stackoverflow.com/questions/69192050/using-selenium-...

  • 6 kudos
3 More Replies
Suman
by New Contributor III
  • 2655 Views
  • 2 replies
  • 2 kudos

Can Lint be implemented in multilingual databricks notebook. ?

I want to implement lint in my project as part of CICD. But as my notebooks has a mix of python and scala . Can Lint be implemented there ?

  • 2655 Views
  • 2 replies
  • 2 kudos
Latest Reply
Suman
New Contributor III
  • 2 kudos

@Vidula Khanna​ I didn't hear from anyone . I am looking for this answer to implement in my databricks code of my project.Can you please assist.

  • 2 kudos
1 More Replies
Labels