cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

unity_Catalog
by New Contributor III
  • 511 Views
  • 1 replies
  • 0 kudos

Concurrent installation of UCX in multiple workspaces

I am trying to install UCX in multiple workspaces concurrently through bash script but facing issue continuously. I have created separate directories for each workspace. I'm facing the below error everytime.Installing UCX in Workspace1Error: lib: cle...

  • 511 Views
  • 1 replies
  • 0 kudos
Latest Reply
Brahmareddy
Honored Contributor
  • 0 kudos

Hi @unity_Catalog, How are you dong today?Try running the UCX installations sequentially to avoid file access conflicts, adding a small delay between each. Ensure each workspace uses a separate installation directory to prevent overlap. You could als...

  • 0 kudos
rohit_kumar
by New Contributor
  • 377 Views
  • 1 replies
  • 0 kudos

Informatica API data retrieval through Datrbicks Workflows or ADF!! Which is better?

                                                              The above set of activities took some 4 hours at ADF to explore and design with greater ease of use, connections, monitoring and it could have probably taken 4 days or more using Databrick...

rohit_kumar_0-1724760571211.png rohit_kumar_1-1724760571240.png rohit_kumar_2-1724760571264.png rohit_kumar_3-1724760571282.png
  • 377 Views
  • 1 replies
  • 0 kudos
Latest Reply
Brahmareddy
Honored Contributor
  • 0 kudos

Hi @rohit_kumar, How are you dong today?To answer to your subject line question, If you're looking for flexibility and integration, Databricks Workflows might be better since it offers native support for complex data transformations and seamless inte...

  • 0 kudos
Haneen_Heeba
by New Contributor III
  • 589 Views
  • 1 replies
  • 0 kudos

Exam suspended

Hello Databricks Team,I had a terrible experience during  certification exam and I have also raised a ticket to the Databricks team but haven’t got any response to the mail till now. I appeared for the Databricks certified Associate Developer for Apa...

  • 589 Views
  • 1 replies
  • 0 kudos
Latest Reply
Haneen_Heeba
New Contributor III
  • 0 kudos

Hi @Cert-Team Could you please look into this issue and assist me in rescheduling my exam since it’s very important for me to provide my certification to my employer at the earliest.Thanks and RegardsHaneen Heeba

  • 0 kudos
sanjay
by Valued Contributor II
  • 735 Views
  • 1 replies
  • 1 kudos

Resolved! Remove duplicate records using pyspark

Hi,I am trying to remove duplicate records from pyspark dataframe and keep the latest one. But somehow df.dropDuplicates["id"] keeps the first one instead of latest. One of the option is to use pandas drop_duplicates, Is there any solution in pyspark...

  • 735 Views
  • 1 replies
  • 1 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 1 kudos

Hi @sanjay ,You can write window function that will rank your rows and then filter rows based on that rank.Take a look on below stackoverflow thread: https://stackoverflow.com/questions/63343958/how-to-drop-duplicates-but-keep-first-in-pyspark-datafr...

  • 1 kudos
Raven91
by New Contributor III
  • 3669 Views
  • 13 replies
  • 4 kudos

Can't activate users

A while back apparently a user became inactive on our Databricks platform from unknown reason.So far everything we tried haven't worked:Delete and manually re-create the userDelete and let SSO to create the user on loginUse Databricks CLI - shows no ...

  • 3669 Views
  • 13 replies
  • 4 kudos
Latest Reply
AsgerLarsen
New Contributor III
  • 4 kudos

I had the same issue on my admin account seamlessly becoming inactive at random.The problem occurred after helping our platform team testing out a new setup of another databricks workspace.I was testing the setup logging in as a standard user and an ...

  • 4 kudos
12 More Replies
vengroff
by New Contributor
  • 733 Views
  • 0 replies
  • 0 kudos

Installation of cluster requirements.txt does not appear to run as google service account

I am running on 15.4 LTS Beta, which supports cluster-level requirements.txt files. The particular requirements.txt I have uploaded to my workspace specifies an extra index URL using a first line that looks like--extra-index-url https://us-central1-p...

  • 733 Views
  • 0 replies
  • 0 kudos
samandrew3
by New Contributor
  • 914 Views
  • 1 replies
  • 0 kudos

Unlocking the Power of Databricks: A Comprehensive Guide for Beginners

In the rapidly evolving world of big data, Databricks has emerged as a leading platform for data engineering, data science, and machine learning. Whether you're a data professional or someone looking to expand your knowledge, understanding Databricks...

  • 914 Views
  • 1 replies
  • 0 kudos
Latest Reply
Rishabh-Pandey
Esteemed Contributor
  • 0 kudos

@samandrew3 keep up the good work 

  • 0 kudos
HelloME
by New Contributor
  • 1549 Views
  • 3 replies
  • 1 kudos

Databricks Asset Bundles - terraform.tfstate not matching when using databricks bundle deploy

Hello I have notice something strange with the asset bundle deployments via the CLI toolI am trying to run databricks bundle deploy and i'm getting and error saying the job ID doesn't exist or I don't have access to it. Error: cannot read job: User h...

  • 1549 Views
  • 3 replies
  • 1 kudos
Latest Reply
joho
New Contributor II
  • 1 kudos

hello,I experienced the same problem today. However i was able to fix it by deleting the tf statefile containing the deleted job. (Located in the .bundle folder in my workspace under the user/ service principal who deployed the bundle. )

  • 1 kudos
2 More Replies
minhngc4795
by New Contributor II
  • 1360 Views
  • 4 replies
  • 1 kudos

How to change the OAuth token lifetime and the maximum number of OAuth tokens

Hi team,I’m working on generating an OAuth token using a service principal, following the instructions here: https://docs.databricks.com/en/dev-tools/auth/oauth-m2m.html#language-CLI, specifically the section on manually generating a workspace-level ...

  • 1360 Views
  • 4 replies
  • 1 kudos
Latest Reply
minhngc4795
New Contributor II
  • 1 kudos

Thank you for your reply @szymon_dybczak service principal added to admins group: This is the problem @szymon_dybczak, we don't want that service principal to get the admin privilege, it should have access to some tables/schemas in our workspace but ...

  • 1 kudos
3 More Replies
Ram9
by Contributor
  • 18959 Views
  • 57 replies
  • 29 kudos

Session expired Error - Databricks Community

Hi Team,        After login into Databricks Community Edition within 2mins I am getting the following Error i.e, " Session expired. Your session has expired, please authenticate again." I have tried to login back again but still it is asking the Auth...

  • 18959 Views
  • 57 replies
  • 29 kudos
Latest Reply
ankitrodekar56
New Contributor II
  • 29 kudos

I am Logging in from INDIA

  • 29 kudos
56 More Replies
Shree23
by New Contributor III
  • 9049 Views
  • 5 replies
  • 1 kudos

databricks vscode profile

Hello,I'm trying to connect to our databricks instance using the vscode extension. However, when following this guide we cannot get the configuration to proceed past the point that it asks for our instance URL.when i am creating profile [DEFAULT]host...

Shree23_0-1698728140170.png
  • 9049 Views
  • 5 replies
  • 1 kudos
Latest Reply
morozg05
New Contributor II
  • 1 kudos

cool

  • 1 kudos
4 More Replies
DangEdmond
by New Contributor
  • 1201 Views
  • 1 replies
  • 0 kudos

Alert Notifications not firing. Inconsistent Success Rate

Hi there,I have a few alerts for a query where if the column is set to >= 1, it will send an email notification to me and 2 others. The 2 individuals are already added as a destination.The query works because the results are populating that column wi...

DangEdmond_0-1688668539844.png DangEdmond_1-1688668576523.png
  • 1201 Views
  • 1 replies
  • 0 kudos
Latest Reply
ashettyajith
New Contributor II
  • 0 kudos

I'm facing the same issue.Don't see anything missing but still no emails even when status of the alert is triggered.@DangEdmond did you find the solution? 

  • 0 kudos
stevieg95
by New Contributor III
  • 5276 Views
  • 9 replies
  • 3 kudos

Unable to login to databricks community edition

Hi, I was using databricks and it was working fine. Im using community edition. Suddenly, it logged me out. I also clicked on forgot password, changed the password and when i tried to login again, i keeps me redirecting to login page without any erro...

stevieg95_0-1711422006387.png
  • 5276 Views
  • 9 replies
  • 3 kudos
Latest Reply
sijopchalil2023
New Contributor II
  • 3 kudos

I am still facing this issue. Is there any update ? It says "Session expired, Log back in" and goes to login window.

  • 3 kudos
8 More Replies
SultanMohiuddin
by New Contributor III
  • 701 Views
  • 2 replies
  • 3 kudos

Session Expires every few seconds after logging into community edition

Session Expires every few seconds after logging into community edition

  • 701 Views
  • 2 replies
  • 3 kudos
Latest Reply
studytowork
New Contributor II
  • 3 kudos

It automatically disconnects and then connects after 1 min, destroying overall concentration.

  • 3 kudos
1 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Top Kudoed Authors