cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

sraj43
by New Contributor II
  • 830 Views
  • 1 replies
  • 2 kudos

Unable to create account in Databricks community Edition

Unable to login to the community edition as the verification email is not delivered.

  • 830 Views
  • 1 replies
  • 2 kudos
Latest Reply
himi1303
New Contributor II
  • 2 kudos

Hi sraj, facing same issue. Is your issue resolved. Please guide me also. That will be of great help.

  • 2 kudos
User16826994223
by Honored Contributor III
  • 3928 Views
  • 2 replies
  • 2 kudos

Mult task - restart of the failed jobs

Hi Team I am using Multitask and I am trying to restart only the failed task but seems like I have to restart complete workflow again and again , is there any way or workaround

  • 3928 Views
  • 2 replies
  • 2 kudos
Latest Reply
TheOptimizer
Contributor
  • 2 kudos

One way that works is to go to your task definition, click advanced options, and set retry policy. The task will restart per those instructions. Does that work for you?

  • 2 kudos
1 More Replies
rbiddle
by New Contributor
  • 6711 Views
  • 4 replies
  • 0 kudos

Specifying a Managed Resource Group name

Azure Databricks provisions a Managed Resource Group when you create your Workspace. Is there a way to specify the name of the Managed Resource Group and its resources during creation?The defaults created by the Workspace violate my company's standar...

  • 6711 Views
  • 4 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hey there @Robert Biddle​ Hope you are well. Just wanted to see if you were able to find an answer to your question and would you like to mark an answer as best? It would be really helpful for the other members too.Cheers!

  • 0 kudos
3 More Replies
Confused
by New Contributor III
  • 3492 Views
  • 5 replies
  • 3 kudos

Resolved! Dealing with updates to a delta table being used as a streaming source

Hi AllI have a requirement to perform updates on a delta table that is the source for a streaming query.I would like to be able to update the table and have the stream continue to work while also not ending up with duplicates.From my research it se...

  • 3492 Views
  • 5 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hey @Mathew Walters​ Hope you are doing great.Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 3 kudos
4 More Replies
JananiMohan
by New Contributor
  • 5266 Views
  • 4 replies
  • 0 kudos

Resolved! ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

After the new release of numpy 1.22.0 on Dec 31st, Databricks failed with this error for my existing Databricks Notebook Version 10.1 and numpy 1.20.0Qn: Why did the earlier releases after 1.20.0 uptil 1.22.0 did not raise the same exception. ?

  • 5266 Views
  • 4 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Janani Mohan​ Hope you are doing well.Just wanted to check in if you were able to resolve your issue or do you need more help? We'd love to hear from you.Thanks!

  • 0 kudos
3 More Replies
KC_1205
by New Contributor III
  • 2815 Views
  • 9 replies
  • 1 kudos

Resolved! Migrating DB from 7.3 LTS to 9.1 LTS

Hi All,I have a code in the dev and production using DB 7.3 LTS. Now, I would like to update the environment to 9.1 LTS as support is going to finish. I have gone through the documentation given in the following link. https://docs.databricks.com/rele...

  • 2815 Views
  • 9 replies
  • 1 kudos
Latest Reply
gmondauto
New Contributor II
  • 1 kudos

@Kiran Chalasani​  Hey Have you ever been able to run 7.3run time with multi_gpus before you migrated to 9.1?

  • 1 kudos
8 More Replies
pantelis_mare
by Contributor III
  • 3478 Views
  • 10 replies
  • 11 kudos

Resolved! Delta Upsert performance on empty table

Hello all,I was just wandering, performance wise how does it compare a plain write operation with a merge operation on an EMPTY delta table. Do we really risk to get significant performance drop?The use case would be to have the same pipeline for ini...

  • 3478 Views
  • 10 replies
  • 11 kudos
Latest Reply
pantelis_mare
Contributor III
  • 11 kudos

Hello @Kaniz Fatma​ ,Unfortunately I did not do any further investigation on the subject. Given that the merge on an empty table will only be done once at the creation of a table, it wouldn't really matter to be honest.

  • 11 kudos
9 More Replies
720677
by New Contributor III
  • 1326 Views
  • 3 replies
  • 2 kudos

Resolved! Databricks Clusters on GCP stop working "Environment directory not found" issue - waitForEnvironmentFileSystem

Starting from yesterday 17/5/2022 i start getting errors while running notebooks or jobs on clusters of Databricks GCP. The error is: SparkException: Environment directory not found at /local_disk0/.ephemeral_nfs/cluster_libraries/pythonThe job/noteb...

  • 1326 Views
  • 3 replies
  • 2 kudos
Latest Reply
720677
New Contributor III
  • 2 kudos

Databricks supports detected an issue with the NFS mounts on GCP. Looks like DBR 10.X versions were affected. After several hours they fixed it and now the same clusters are back to normal.

  • 2 kudos
2 More Replies
User16826994223
by Honored Contributor III
  • 838 Views
  • 1 replies
  • 0 kudos

Some of the limitation I see In docs of photon until now july 2021 is  Works on Delta and Parquet tables only for both read and write.Does not suppor...

Some of the limitation I see In docs of photon until now july 2021 is Works on Delta and Parquet tables only for both read and write.Does not support the following data types:MapArrayDoes not support window and sort operatorsDoes not support Spark S...

  • 838 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Excellent post, Thank you @Kunal Gaurav​ !

  • 0 kudos
sparkstreaming
by New Contributor III
  • 4973 Views
  • 7 replies
  • 6 kudos

Resolved! Rest API invocation for databricks notebook fails while invoking from ADF pipeline

In the current implementation a streaming databricks notebook needs to be started based on the configuration passed. Since the rest of databricks notebooks are being invoked by using ADF,it was decided to use ADF for starting these notebooks. Since t...

  • 4973 Views
  • 7 replies
  • 6 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 6 kudos

Hi @Prasanth KP​ , Just a friendly follow-up. Do you still need help, or @Hubert Dudek (Customer)​ and @Werner Stinckens​ 's responses help you to find the solution? Please let us know.

  • 6 kudos
6 More Replies
Anuj93
by New Contributor III
  • 702 Views
  • 2 replies
  • 0 kudos

Mount Point Service Principle

Is there any way to find out the service principle/Azure Active Directory applicationused to mount a container?

  • 702 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @Anuj Kumar​ , Just a friendly follow-up. Do you still need help, or does the above response help you to find the solution? Please let us know.

  • 0 kudos
1 More Replies
kiselevanton
by New Contributor II
  • 788 Views
  • 2 replies
  • 3 kudos

Community edition, sign up issue

I am studying Databricks as part of Data Science with Databricks for Data Analysts Specialization. It requires me to sign up for community edition, but it simply doesn't work.I have tried to sign up 3 times with 3 different e-mails, 2 private and 1 c...

  • 788 Views
  • 2 replies
  • 3 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 3 kudos

Hi @Anton Kiselev​, Our sincere apologies for the delayed response; it won't happen again.We had a temporary bug that affected a small number of users; our regrets if you were impacted. We have fixed the bug and your account should now be back to nor...

  • 3 kudos
1 More Replies
alejandrofm
by Valued Contributor
  • 1454 Views
  • 2 replies
  • 5 kudos

Resolved! How to set a global checkpoint for all notebooks?

I have several users doing data analysis on Databricks Spark notebooks, everything is smooth, now I want to make sure that the checkpointdir is configured on the cluster start, so every user doesn't had to set it on the Notebook (ending up in a lot o...

image
  • 1454 Views
  • 2 replies
  • 5 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 5 kudos

Hi @Alejandro Martinez​ , Just a friendly follow-up. Do you still need help, or @Hubert Dudek (Customer)​ 's response help you to find the solution? Please let us know.

  • 5 kudos
1 More Replies
Cameron_Afzal
by New Contributor II
  • 764 Views
  • 2 replies
  • 0 kudos

I'm unable to create an account for Databricks Community Edition. I've tried multiple email addresses and browsers across multiple attempts. I...

I'm unable to create an account for Databricks Community Edition. I've tried multiple email addresses and browsers across multiple attempts. I fill out and submit the sign-up form but never receive the email and thus can't log in. Any advice? Are the...

  • 764 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @Cameron Afzal​  and @tipu sultan​ , Thank you for reaching out! Our sincere apologies for the delayed response; it won't happen again.We had a temporary bug that affected a small number of users; our regrets if you were impacted. We have fixed th...

  • 0 kudos
1 More Replies
prasadvaze
by Valued Contributor II
  • 5494 Views
  • 5 replies
  • 5 kudos

Resolved! Limit on number of result rows displayed on databricks SQL UI

Databricks SQL UI currently limits the query results display to 64000 rows. When will this limit go away? Using SSMS I get 40MM rows results in the UI and my users won't switch to databricks SQL for this reason

  • 5494 Views
  • 5 replies
  • 5 kudos
Latest Reply
User16765136105
New Contributor III
  • 5 kudos

Hi @prasad vaze​ - We do have a feature in the works that will increase this limit. If you reach out to your Databricks contact they can give you more details regarding dates and the preview.

  • 5 kudos
4 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels
Top Kudoed Authors