cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Engineering & Streaming

Forum Posts

Start a conversation
rahul3
by • New Contributor
  • 2499 Views
  • 2 replies
  • 1 kudos

Facing mount/unmount issue while running same job parallelly with scala.

 Using above configuration in cluster, when I run databricks job parallelly with multiple request at a same time, then I am getting mount/unmount issue. For an example : When I make three request to databricks job , it run 3 jobs parallelly but somet...

image.png image.png
Data Engineering
Reply
  • 2499 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

Hi @rahul upadhyay​, We haven’t heard from you on the last response from @Prabakar Ammeappin​ . If you have any solution, please share it with the community as it can be helpful to others. Otherwise, we will respond with more details and try to help.

  • 1 kudos
1 More Replies
gazzyjuruj
by • Contributor II
  • 3022 Views
  • 6 replies
  • 4 kudos

User Initiated Cluster Termination, after 1-2 hours cluster goes offline even while working!

Hi,Wanted to ask, I have been experiencing databricks_error = user initiated cluster terminationdatabricks_error_message: timeout while placing nodesand soetimes it stays stuck on "finding new nodes, instances if neccessary" for hours..and it goes of...

Data Engineering
Reply
  • 3022 Views
  • 6 replies
  • 4 kudos
View Replies
Latest Reply
Kaniz_Fatma
Community Manager
  • 4 kudos

Hi @Ghazanfar Uruj​  , We haven’t heard from you on the last response from @Atanu Sarkar​, and I was checking back to see if his suggestions helped you. Or else, If you have any solution, please share it with the community as it can be helpful to oth...

  • 4 kudos
5 More Replies
Mr__E
by • Contributor II
  • 1702 Views
  • 3 replies
  • 1 kudos

Databricks dashboard removing order, incorrectly sorted.

I created a table that aggregates data by year and week of year and display this in a chart over time. As part of the query, I order by the year, then week columns. In the visualization on the query (in the SQL editor), I disabled the sort, because i...

Data Engineering
Reply
  • 1702 Views
  • 3 replies
  • 1 kudos
View Replies
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

Hi @Erik Louie​ , We haven’t heard from you on the last response from me, and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpful to others. Otherwise, we will res...

  • 1 kudos
2 More Replies
Manav
by • New Contributor II
  • 1223 Views
  • 2 replies
  • 1 kudos

I am unable to create cluster due to unexpected launch failure. I have attached the image with full description of the error. Please help!

Data Engineering
Reply
  • 1223 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

Hi @Manav Mishra​, We haven’t heard from you on the last response from me, and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpful to others. Otherwise, we will re...

  • 1 kudos
1 More Replies
MMMM
by • New Contributor III
  • 10097 Views
  • 15 replies
  • 2 kudos

cluster issue

I signed up with databricks free trial but I am not be able to create new cluster , could you help me in this regard ?When i create cluster nothing happens and it terminates automatically , I'm on aws

Data Engineering
Reply
  • 10097 Views
  • 15 replies
  • 2 kudos
View Replies
Latest Reply
MMMM
New Contributor III
  • 2 kudos

Could you help in answer all my questions 1 by one again

  • 2 kudos
14 More Replies
Dicer
by • Valued Contributor
  • 2929 Views
  • 5 replies
  • 5 kudos

Resolved! Azure Databricks: AnalysisException: Database 'bf' not found

I wanted to save my delta tables in my Databricks database. When I saveAsTable, there is an error message Azure Databricks: AnalysisException: Database 'bf' not found​Ye, There is no database named "bf" in my database.Here is my full code:import os i...

Data Engineering
Reply
  • 2929 Views
  • 5 replies
  • 5 kudos
View Replies
Latest Reply
Dicer
Valued Contributor
  • 5 kudos

Some data can be saved as delta tables while some cannot.

  • 5 kudos
4 More Replies
mani238
by • New Contributor III
  • 13517 Views
  • 3 replies
  • 3 kudos

Resolved! Run result unavailable: job failed with error message Library installation failed for library due to user error for jar: "dbfs:/my-jar.jar"

Run result unavailable: job failed with error message Library installation failed for library due to user error for jar: "dbfs:/my-jar.jar" . Error messages: Library installation attempted on the driver node of cluster 0510-013936-3cc6d9kw and failed...

Data Engineering
Reply
  • 13517 Views
  • 3 replies
  • 3 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hey there @manivannan p​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from yo...

  • 3 kudos
2 More Replies
Dataminion
by • New Contributor
  • 3042 Views
  • 2 replies
  • 3 kudos

Resolved! No module named dlt

Cluster version 10.5 on AWS.Trying to import dlt gave error above.​Am I missing something?​

Data Engineering
Reply
  • 3042 Views
  • 2 replies
  • 3 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hey there @Cathy Low​ Does @Kaniz Fatma​ 's response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly? Else please let us know if you need more help. We'd love to hear from y...

  • 3 kudos
1 More Replies
SailajaB
by • Valued Contributor III
  • 3621 Views
  • 2 replies
  • 6 kudos

Resolved! How to pass a variable which holds a value to child notebook using run command

Hello,We have 3 notebooks as below. And trying to send a variable where we pass the value to it via job to child notebook with %run commandNotebooks:notebook_par,notebook_child1 and notebook_child2.In parent note book we are calling notebook_child1 u...

Data Engineering
Reply
  • 3621 Views
  • 2 replies
  • 6 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 6 kudos

Hey there @Sailaja B​ How's it going?Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Chee...

  • 6 kudos
1 More Replies
prasadvaze
by • Valued Contributor II
  • 33142 Views
  • 11 replies
  • 9 kudos

When to use delta lake versus relational database as a source for BI reporting?

Assume all of your data exists in delta tables and also in SQL server so you have a choice to report from either. Can someone share thoughts on "In what scenario you would not want report created from delta table and instead use the traditional rel...

Data Engineering
Reply
  • 33142 Views
  • 11 replies
  • 9 kudos
View Replies
Latest Reply
PCJ
New Contributor II
  • 9 kudos

Hi @Kaniz Fatma​  - I would like a follow-up on @prasad vaze​ question regarding unsupported referential integrity. How does one work around that, using best practices as Databricks sees it?

  • 9 kudos
10 More Replies
Tahseen0354
by • Valued Contributor
  • 20492 Views
  • 8 replies
  • 5 kudos

Resolved! Getting "Job aborted due to stage failure" SparkException when trying to download full result

I have generated a result using SQL. But whenever I try to download the full result (1 million rows), it is throwing SparkException. I can download the preview result but not the full result. Why ? What happens under the hood when I try to download ...

Data Engineering
Reply
  • 20492 Views
  • 8 replies
  • 5 kudos
View Replies
Latest Reply
rpshgupta
New Contributor III
  • 5 kudos

I am also having this issue again and again. I really want to understand what can we do to avoid this?

  • 5 kudos
7 More Replies
Steveo
by • New Contributor II
  • 967 Views
  • 3 replies
  • 0 kudos

In Databricks, when saving a query, the changes do no persist. Any advice?

When returning to the query, the changes have reverted.Is this because the query is open by another user?How can this be resolved? #databricks

Data Engineering
Reply
  • 967 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
Steveo
New Contributor II
  • 0 kudos

We ARE using the save feature. But we are still loosing changes. Is this because we have multiple Databricks tabs open in separate Chrome browsers? Across 2 users.

  • 0 kudos
2 More Replies
gazzyjuruj
by • Contributor II
  • 14022 Views
  • 14 replies
  • 11 kudos

Resolved! Failed to start cluster

Hi, I ran the cluster more than 5-6 times with it failing to start since this past morning (about 11-12 hours now) since i'm facing this problem.Attaching screenshot below and also typing in case someone comes from the web to this thread in future.Pr...

IMG_2152 IMG_2151
Data Engineering
Reply
  • 14022 Views
  • 14 replies
  • 11 kudos
View Replies
Latest Reply
worthpicker
New Contributor II
  • 11 kudos

The cluster will be able to start and the nodes will automatically obtain the updated cluster configuration data.

  • 11 kudos
13 More Replies
Anonymous
by • Not applicable
  • 1110 Views
  • 0 replies
  • 1 kudos

7 Steps For Mobile Banking App Development Banking industry experts have predicted that as soon as the pandemic is over, the banking institutions will...

7 Steps For Mobile Banking App DevelopmentBanking industry experts have predicted that as soon as the pandemic is over, the banking institutions will have to reform hooeyapps their operations by inclining towards digital banking. In the recent past, ...

Data Engineering
Reply
  • 1110 Views
  • 0 replies
  • 1 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels
Top Kudoed Authors
View all