cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

sanjay
by Valued Contributor II
  • 18783 Views
  • 4 replies
  • 3 kudos

Resolved! How to backup databrick delta tables or workspace

Hi,I am trying to understand how to take backup of databrick delta tables/workspace and how to restore in case if any failure.Or suggest me any alternative solution to revert back if data is corrupted.Regards,Sanjay

  • 18783 Views
  • 4 replies
  • 3 kudos
Latest Reply
NandiniN
Databricks Employee
  • 3 kudos

Hi @Sanjay Jain​ ,Here are some of the ways Deep Clone: https://www.databricks.com/wp-content/uploads/notebooks/using-deep-clone-disaster-recovery-delta-lake-databricks.htmlRepos for notebooks and code: https://docs.databricks.com/repos/index.htmlht...

  • 3 kudos
3 More Replies
KVNARK
by Honored Contributor II
  • 4043 Views
  • 7 replies
  • 7 kudos

Resolved! No certification vouchers in Rewards portal.

there are no certification vouchers in rewards portal. Could some one from data bricks team update whether you are planning to include them again and also very few data bricks merchandise.

  • 4043 Views
  • 7 replies
  • 7 kudos
Latest Reply
Anonymous
Not applicable
  • 7 kudos

Hi @KVNARK .​ Thank you for reaching out to us!The reward store will be updated for the Spring and new rewards will be introduced in the coming weeks. Kindly, bear with us.Regards

  • 7 kudos
6 More Replies
mkd_140848
by New Contributor II
  • 3771 Views
  • 5 replies
  • 0 kudos

Open notebook in new tab not working as of 2 March 2023. Web browser: Google Chrome 110.0.5481.178 Databricks version: Standard Cloud platform: Azure

Before Feb 2023, I often opened Notebook in new tabs by Ctrl (or Command for Mac) + Click the notebook names, in the Workspace explorer UI. Recently it stopped working (Ctrl + Click Notebook name, nothing happened). Is this a bug?

  • 3771 Views
  • 5 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @David H​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 0 kudos
4 More Replies
Kayla
by Valued Contributor II
  • 3722 Views
  • 2 replies
  • 0 kudos

BigQuery - Delete or update from Databricks?

I'm trying to sync a delta table in Databricks to a BigQuery table. For the most part, appending is sufficient, but occassionally we need to overwrite rows - which we've only been able to do by overwriting the entire table.Is there any way to do upda...

  • 3722 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Kayla Pomakoy​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...

  • 0 kudos
1 More Replies
souvik
by New Contributor
  • 1209 Views
  • 2 replies
  • 0 kudos

Unable to get or download fundamentals of databricks lake house platform badge download

Hi Team,I have completed the fundamentals of databricks lakehosue platform and received the certificate but unable to download or get the badge.When I login to website I am unbale to see the badge as well.Please help me on this.I haven't received any...

  • 1209 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Souvik Banerjee​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...

  • 0 kudos
1 More Replies
RajeshRK
by Contributor II
  • 5026 Views
  • 7 replies
  • 2 kudos

How to optimize jobs performance

Hi Team,We have a complex ETL job running in databricks for 6 hours. The cluster has the below configuration: Minworkers: 16Maxworkers: 24Worker and Driver Node Type: Standard_DS14_v2. (16 cores, 128 GB RAM)I have monitored the job progress in Spark...

  • 5026 Views
  • 7 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Rajesh Kannan R​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedb...

  • 2 kudos
6 More Replies
AyushModi038
by New Contributor III
  • 5152 Views
  • 6 replies
  • 0 kudos

Library mismatch in same cluster different file

In continuation to the issues encountered in this discussion.https://community.databricks.com/s/feed/0D58Y00009tCiQTSA0 I have a bizzare issue.Here are the 2 screenshots taken few seconds apart1.2 . Same cluster, same command, executed 6 seconds apar...

image image
  • 5152 Views
  • 6 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Ayush Modi​  I'm sorry you could not find a solution to your problem in the answers provided.Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.I suggest pr...

  • 0 kudos
5 More Replies
Phani1
by Valued Contributor II
  • 2028 Views
  • 2 replies
  • 3 kudos

Efficiently orchestrate data bricks jobs

Hi Team,How efficiently can orchestrate data bricks jobs which involve a lot of transformations, dependencies, and complexity?At source have a lot of SSIS packages that have complex dependencies and more transformation.     We have the following opti...

  • 2028 Views
  • 2 replies
  • 3 kudos
Latest Reply
Phani1
Valued Contributor II
  • 3 kudos

My question is, how do we reliably orchestrate multiple Databricks Jobs/Workflows that are running in a mixed latency and can write to the same silver and gold delta tables? Could you please suggest the best approach and practices for the same?

  • 3 kudos
1 More Replies
lawrence009
by Contributor
  • 5439 Views
  • 6 replies
  • 4 kudos

Git Integration: selective check-in not working

We started noticing the problem about 3 weeks ago. Databricks' Git GUI fails to commit only the selected files. All the files that have been modified, added or removed since that last commit will get checked in even if it de-selected in the GUI.Has ...

  • 5439 Views
  • 6 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Lawrence Chen​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...

  • 4 kudos
5 More Replies
manasa
by Contributor
  • 8109 Views
  • 4 replies
  • 2 kudos

org.apache.spark.SparkException: Job aborted due to stage failure: Authorized committer failed while pushing dataframe to azure cosmos db.

I am writing data to the azure cosmos db using OLTP connector using below codecfg["spark.cosmos.write.strategy"]="ItemOverwrite" json_df.write.format("cosmos.oltp").options(**cfg).mode("APPEND").save()I am getting below error Please let me know i...

image.png image.png
  • 8109 Views
  • 4 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Manasa Kalluri​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedba...

  • 2 kudos
3 More Replies
Lewis_Wong
by New Contributor II
  • 2307 Views
  • 4 replies
  • 2 kudos

Resolved! Adding tags to jobs from Tableau / Python (ODBC)

Hi all,We are using Azure Databricks.We would like to see if we can track usage of jobs initiated from Tableau /Python (via simba spark ODBC driver) One way we can think of is to add tag to the job. But we are not sure if we can add tags to job when ...

  • 2307 Views
  • 4 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@Vidula Khanna​ We can add custom values to the job clusters as well. When you create the job task, click on edit cluster details which will take you to the cluster configuration where you can add the custom tags. In case of Tableau queries, it would...

  • 2 kudos
3 More Replies
Paradox_Parijat
by New Contributor III
  • 4376 Views
  • 7 replies
  • 4 kudos

Didn't receive Databricks badge

I just completed the Databricks Lakehouse Fundamentals exam, but while redirecting to credentials.databricks.com. I'm not able to find any badge provided to me for completion. Can you help me out? @Vidula Khanna​ 

  • 4376 Views
  • 7 replies
  • 4 kudos
Latest Reply
sher
Valued Contributor II
  • 4 kudos

check your mail. yesterday I have completed they shared me through registered email

  • 4 kudos
6 More Replies
Mafofola
by New Contributor II
  • 1442 Views
  • 3 replies
  • 0 kudos

Am using databricks trying to login​

Am using databricks trying to login​

  • 1442 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Ige Makanjuola​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...

  • 0 kudos
2 More Replies
ashish577
by New Contributor III
  • 7330 Views
  • 5 replies
  • 5 kudos

Is it not possible to create multiple tables on same location in unity catalog?

I have tried 2 tables with different catalogs, schemas, column names and it throws LOCATION_OVERLAP. In spark we can create as many tables as needed on the same location.

  • 7330 Views
  • 5 replies
  • 5 kudos
Latest Reply
Anonymous
Not applicable
  • 5 kudos

Hi @Ashish Singh​ I'm sorry you could not find a solution to your problem in the answers provided.Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.I suggest p...

  • 5 kudos
4 More Replies
Alyayman
by Contributor
  • 2149 Views
  • 3 replies
  • 3 kudos

Level of databricks spark certificate exam

I am willing to take spark certificate exam , i have solved the practice test and it was very good , but i am afraid that the level of exam is harder than the practice exam . So does it really harder ?

  • 2149 Views
  • 3 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Aly Ayman​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers you...

  • 3 kudos
2 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels