After the new release of numpy 1.22.0 on Dec 31st, Databricks failed with this error for my existing Databricks Notebook Version 10.1 and numpy 1.20.0Qn: Why did the earlier releases after 1.20.0 uptil 1.22.0 did not raise the same exception. ?
Hi @Janani Mohan Hope you are doing well.Just wanted to check in if you were able to resolve your issue or do you need more help? We'd love to hear from you.Thanks!
Hi All,I have a code in the dev and production using DB 7.3 LTS. Now, I would like to update the environment to 9.1 LTS as support is going to finish. I have gone through the documentation given in the following link. https://docs.databricks.com/rele...
Hello all,I was just wandering, performance wise how does it compare a plain write operation with a merge operation on an EMPTY delta table. Do we really risk to get significant performance drop?The use case would be to have the same pipeline for ini...
Hello @Kaniz Fatma ,Unfortunately I did not do any further investigation on the subject. Given that the merge on an empty table will only be done once at the creation of a table, it wouldn't really matter to be honest.
Starting from yesterday 17/5/2022 i start getting errors while running notebooks or jobs on clusters of Databricks GCP. The error is: SparkException: Environment directory not found at /local_disk0/.ephemeral_nfs/cluster_libraries/pythonThe job/noteb...
Databricks supports detected an issue with the NFS mounts on GCP. Looks like DBR 10.X versions were affected. After several hours they fixed it and now the same clusters are back to normal.
Some of the limitation I see In docs of photon until now july 2021 is Works on Delta and Parquet tables only for both read and write.Does not support the following data types:MapArrayDoes not support window and sort operatorsDoes not support Spark S...
In the current implementation a streaming databricks notebook needs to be started based on the configuration passed. Since the rest of databricks notebooks are being invoked by using ADF,it was decided to use ADF for starting these notebooks. Since t...
Hi @Prasanth KP , Just a friendly follow-up. Do you still need help, or @Hubert Dudek (Customer) and @Werner Stinckens 's responses help you to find the solution? Please let us know.
I am studying Databricks as part of Data Science with Databricks for Data Analysts Specialization. It requires me to sign up for community edition, but it simply doesn't work.I have tried to sign up 3 times with 3 different e-mails, 2 private and 1 c...
Hi @Anton Kiselev, Our sincere apologies for the delayed response; it won't happen again.We had a temporary bug that affected a small number of users; our regrets if you were impacted. We have fixed the bug and your account should now be back to nor...
I have several users doing data analysis on Databricks Spark notebooks, everything is smooth, now I want to make sure that the checkpointdir is configured on the cluster start, so every user doesn't had to set it on the Notebook (ending up in a lot o...
Hi @Alejandro Martinez , Just a friendly follow-up. Do you still need help, or @Hubert Dudek (Customer) 's response help you to find the solution? Please let us know.
I'm unable to create an account for Databricks Community Edition. I've tried multiple email addresses and browsers across multiple attempts. I fill out and submit the sign-up form but never receive the email and thus can't log in. Any advice? Are the...
Hi @Cameron Afzal and @tipu sultan , Thank you for reaching out! Our sincere apologies for the delayed response; it won't happen again.We had a temporary bug that affected a small number of users; our regrets if you were impacted. We have fixed th...
Databricks SQL UI currently limits the query results display to 64000 rows. When will this limit go away? Using SSMS I get 40MM rows results in the UI and my users won't switch to databricks SQL for this reason
Hi @prasad vaze - We do have a feature in the works that will increase this limit. If you reach out to your Databricks contact they can give you more details regarding dates and the preview.
We have a Databricks instance on Azure that has somewhat organically grow with dozens of users and hundreds of notebooks. How do I conveniently backup this env so in case disaster strikes the notebooks aren't lost? The data itself is backed by Azure ...
I have a scheduled a notebook. can I keep current date as default in widget whenever the notebook run and also i need the flexibility to change the widget value to any other date based on the ad hoc run that I do.
Hi @philip george , Just a friendly follow-up. Do you still need help, or @Hubert Dudek (Customer) 's and @Werner Stinckens's response help you to find the solution? Please let us know.
As title, I need to clone code from my private git repo, and use it in my notebook, I do something likedef cmd(command, cwd=None):
process = subprocess.Popen(command.split(), stdout=subprocess.PIPE, cwd=cwd)
output, error = process.communicate(...
Hi @Andy Huang , Just a friendly follow-up. Do you still need help, or @Prabakar Ammeappin 's response help you to find the solution? Please let us know.
