cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

unity_Catalog
by New Contributor III
  • 930 Views
  • 1 replies
  • 0 kudos

Concurrent installation of UCX in multiple workspaces

I am trying to install UCX in multiple workspaces concurrently through bash script but facing issue continuously. I have created separate directories for each workspace. I'm facing the below error everytime.Installing UCX in Workspace1Error: lib: cle...

  • 930 Views
  • 1 replies
  • 0 kudos
Latest Reply
Brahmareddy
Esteemed Contributor
  • 0 kudos

Hi @unity_Catalog, How are you dong today?Try running the UCX installations sequentially to avoid file access conflicts, adding a small delay between each. Ensure each workspace uses a separate installation directory to prevent overlap. You could als...

  • 0 kudos
rohit_kumar
by New Contributor
  • 1499 Views
  • 1 replies
  • 0 kudos

Informatica API data retrieval through Datrbicks Workflows or ADF!! Which is better?

                                                              The above set of activities took some 4 hours at ADF to explore and design with greater ease of use, connections, monitoring and it could have probably taken 4 days or more using Databrick...

rohit_kumar_0-1724760571211.png rohit_kumar_1-1724760571240.png rohit_kumar_2-1724760571264.png rohit_kumar_3-1724760571282.png
  • 1499 Views
  • 1 replies
  • 0 kudos
Latest Reply
Brahmareddy
Esteemed Contributor
  • 0 kudos

Hi @rohit_kumar, How are you dong today?To answer to your subject line question, If you're looking for flexibility and integration, Databricks Workflows might be better since it offers native support for complex data transformations and seamless inte...

  • 0 kudos
Haneen_Heeba
by New Contributor III
  • 1039 Views
  • 1 replies
  • 0 kudos

Exam suspended

Hello Databricks Team,I had a terrible experience during  certification exam and I have also raised a ticket to the Databricks team but haven’t got any response to the mail till now. I appeared for the Databricks certified Associate Developer for Apa...

  • 1039 Views
  • 1 replies
  • 0 kudos
Latest Reply
Haneen_Heeba
New Contributor III
  • 0 kudos

Hi @Cert-Team Could you please look into this issue and assist me in rescheduling my exam since it’s very important for me to provide my certification to my employer at the earliest.Thanks and RegardsHaneen Heeba

  • 0 kudos
sanjay
by Valued Contributor II
  • 2150 Views
  • 1 replies
  • 1 kudos

Resolved! Remove duplicate records using pyspark

Hi,I am trying to remove duplicate records from pyspark dataframe and keep the latest one. But somehow df.dropDuplicates["id"] keeps the first one instead of latest. One of the option is to use pandas drop_duplicates, Is there any solution in pyspark...

  • 2150 Views
  • 1 replies
  • 1 kudos
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 1 kudos

Hi @sanjay ,You can write window function that will rank your rows and then filter rows based on that rank.Take a look on below stackoverflow thread: https://stackoverflow.com/questions/63343958/how-to-drop-duplicates-but-keep-first-in-pyspark-datafr...

  • 1 kudos
Phani1
by Valued Contributor II
  • 1619 Views
  • 1 replies
  • 0 kudos

best practices for implementing early arriving fact handling

 Hi All,Can you please share us the best practices for implementing early arriving fact handling in databricks for streaming data processed in near real time using structured streaming.There are many ways to handle this use case in batch/mini batch. ...

Phani1_0-1724754033290.png
  • 1619 Views
  • 1 replies
  • 0 kudos
Latest Reply
Phani1
Valued Contributor II
  • 0 kudos

Greetings Team, I would like to inquire if any of you have suggestions regarding the query.

  • 0 kudos
lean-ai
by New Contributor
  • 875 Views
  • 0 replies
  • 0 kudos

Any women in data+ai interested in gathering to form some a casual networking group based in Sydney?

Hi all,Inspired by the Women in Data+AI Breakfast at the Databricks World Tour in Sydney, I'm considering the potential for a new womens' networking community based in Sydney. This group would cater to women currently working in or interested in purs...

  • 875 Views
  • 0 replies
  • 0 kudos
Raven91
by New Contributor III
  • 6945 Views
  • 13 replies
  • 4 kudos

Can't activate users

A while back apparently a user became inactive on our Databricks platform from unknown reason.So far everything we tried haven't worked:Delete and manually re-create the userDelete and let SSO to create the user on loginUse Databricks CLI - shows no ...

  • 6945 Views
  • 13 replies
  • 4 kudos
Latest Reply
AsgerLarsen
New Contributor III
  • 4 kudos

I had the same issue on my admin account seamlessly becoming inactive at random.The problem occurred after helping our platform team testing out a new setup of another databricks workspace.I was testing the setup logging in as a standard user and an ...

  • 4 kudos
12 More Replies
vengroff
by New Contributor II
  • 1972 Views
  • 0 replies
  • 1 kudos

Installation of cluster requirements.txt does not appear to run as google service account

I am running on 15.4 LTS Beta, which supports cluster-level requirements.txt files. The particular requirements.txt I have uploaded to my workspace specifies an extra index URL using a first line that looks like--extra-index-url https://us-central1-p...

  • 1972 Views
  • 0 replies
  • 1 kudos
VReddy1601
by New Contributor
  • 1091 Views
  • 0 replies
  • 0 kudos

Apply Changes Error in DLT Pipeline

Hi Team, Am Trying to use Apply changes from Bronze to Silver using the below.                   @dlt.table(           name="Silver_Orders",           comment="This table - hive_metastore.silver.Orders reads data from the Bronze layer and writes it t...

  • 1091 Views
  • 0 replies
  • 0 kudos
samandrew3
by New Contributor
  • 2310 Views
  • 1 replies
  • 0 kudos

Unlocking the Power of Databricks: A Comprehensive Guide for Beginners

In the rapidly evolving world of big data, Databricks has emerged as a leading platform for data engineering, data science, and machine learning. Whether you're a data professional or someone looking to expand your knowledge, understanding Databricks...

  • 2310 Views
  • 1 replies
  • 0 kudos
Latest Reply
Rishabh-Pandey
Esteemed Contributor
  • 0 kudos

@samandrew3 keep up the good work 

  • 0 kudos
jmbabazi
by New Contributor II
  • 3644 Views
  • 5 replies
  • 1 kudos

Resolved! Unable to connect to SQL Warehouse from RStudio using U2M authentication

Hi Community,Has anyone encountered the error below, and If yes, how did you solve it? My workspace is setup normally in UC. My workspace also has private endpoint setup. I am authentication over the browser when I run my R code, then right after the...

  • 3644 Views
  • 5 replies
  • 1 kudos
Latest Reply
jmbabazi
New Contributor II
  • 1 kudos

Thanks everyone for the replies. It turned out that this is a known bug and they are working on a fix

  • 1 kudos
4 More Replies
HelloME
by New Contributor
  • 3720 Views
  • 3 replies
  • 1 kudos

Databricks Asset Bundles - terraform.tfstate not matching when using databricks bundle deploy

Hello I have notice something strange with the asset bundle deployments via the CLI toolI am trying to run databricks bundle deploy and i'm getting and error saying the job ID doesn't exist or I don't have access to it. Error: cannot read job: User h...

  • 3720 Views
  • 3 replies
  • 1 kudos
Latest Reply
joho
New Contributor II
  • 1 kudos

hello,I experienced the same problem today. However i was able to fix it by deleting the tf statefile containing the deleted job. (Located in the .bundle folder in my workspace under the user/ service principal who deployed the bundle. )

  • 1 kudos
2 More Replies
venicoldgelcije
by New Contributor
  • 1101 Views
  • 0 replies
  • 0 kudos

Venicold G@l: cijena i učinkovitost liječenja proširenih vena u Croatia

Venicold G@L pomaže brzom zacjeljivanju krvnih žila nogu. Ovaj neinvazivni, brzodjelujući lijek razbija krvne ugruške koji smanjuju krvarenje, uzrokuju otežan protok krvi, tanje stijenke krvnih žila, povećanu viskoznost krvi i eventualno začepljenje ...

  • 1101 Views
  • 0 replies
  • 0 kudos
Ankita04
by New Contributor II
  • 451 Views
  • 0 replies
  • 0 kudos

DataBricks Certification Exam Got Suspended.

Hello , I was taking the Databricks Certified Data Engineer Associate exam when a pop-up appeared, warning me not to look away from the screen. However, I was focused solely on the screen and did not violate any exam rules.Afterwards , the proctor as...

  • 451 Views
  • 0 replies
  • 0 kudos

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels