- 8124 Views
- 8 replies
- 6 kudos
Job aborted due to stage failure: Task 1863 in stage 10.0 failed 4 times, most recent failure: Lost task 1863.3 in stage 10.0 (TID 2021) (10.0.4.7 executor 2): org.apache.spark.SparkException: Python worker exited unexpectedly (crashed): Fatal Python erro
I am getting below error some time run my databricks notebook from ADF, If the executor node is one then it works fine, if it increases 2 or more some times its failing on same data.Cluster Detail : Standard_F4s_v2 · Workers: Standard_F4s_v2 · 1-8 wo...
- 8124 Views
- 8 replies
- 6 kudos
- 6 kudos
Hi @Ancil P A​ Can you give paste the complete stacktrace from the failed task (from failed stage 10.0) and the code snippet that you are trying to run in the notebook . Also, do you think you can raise a databricks support ticket for the same?
- 6 kudos
- 3190 Views
- 1 replies
- 0 kudos
Loading multiple gz files from ADLS to Delta Lake/Delta table in ADB
I have several gz files (file.csv.gz) in a ADLS folder which are of pretty big size. All of these files are extractd from the same base table so it has the similar data but of different dates. How can I transfer them in delta lake/delta table. We wou...
- 3190 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi, You can read GZ files through spark. https://stackoverflow.com/questions/42761912/how-to-read-gz-compressed-file-by-pysparkPlease let us know if this helps. Also, please tag @Debayan​ with your next comment so that I will get notified. Thank you!
- 0 kudos
- 6432 Views
- 3 replies
- 0 kudos
How to filter the Spark UI for a notebook
When running spark under yarn each script has it's own self contained set of logs:- In databricks all I see if a list of jobs and stages that have been run on the cluster:- From a support perspective this is a nightmare.How can notebooks logs be grou...
- 6432 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Dean Lovelace​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers...
- 0 kudos
- 10832 Views
- 4 replies
- 0 kudos
Function returns UNSUPPORTED_CORRELATED_SCALAR_SUBQUERY
Hello experts. The below function in Databricks gives UNSUPPORTED_CORRELATED_SCALAR_SUBQUERY error. We didn't have this issue though in Oracle. Is this a limitation of Databricks? Just to note the final result returns only one row. Thank you in advan...
- 10832 Views
- 4 replies
- 0 kudos
- 0 kudos
Hello @Suteja Kanuri​ can we have any respond on the above?Thank you.
- 0 kudos
- 1653 Views
- 1 replies
- 0 kudos
community.cloud.databricks.com
https://community.cloud.databricks.com/Last night I am still able to use it. This morning breaks down totally, and I could not login. Please help.
- 1653 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @THIAM HUAT TAN​ Thank you for reaching out, and we’re sorry to hear about this log-in issue! We have this Community Edition login troubleshooting post on Community. Please take a look, and follow the troubleshooting steps. If the steps do not res...
- 0 kudos
- 12264 Views
- 5 replies
- 4 kudos
Resolved! Databricks Certification Voucher Code not received
I have an exam to take for the Databricks Associate ML certification early this week. I have raised multiple tickets for the same and previously but didn't receive any reply on the same. I have attended the webinar for 3 days on Data Engineering as m...
- 12264 Views
- 5 replies
- 4 kudos
- 4 kudos
Hi @Avinash Tiwari​ I hope you are doing great!I have forwarded your query to our Academic team. Soon the problem will be resolved. Please bear with us.Thanks and Regards
- 4 kudos
- 17470 Views
- 3 replies
- 1 kudos
Root cause analysis and Woraround for "Error: AttributeError: type object 'Retry' has no attribute 'DEFAULT_METHOD_WHITELIST'&quo...
Root cause analysis and Woraround for "Error: AttributeError: type object 'Retry' has no attribute 'DEFAULT_METHOD_WHITELIST'"---------------------------------------------ProblemWhen using databricks-cli, starting with 4th of May this error occurs:Er...
- 17470 Views
- 3 replies
- 1 kudos
- 1 kudos
For what its worth I can confirm the pinning works. I also pinned the requests package. See my details here as I am using pipelines to handle deployment. I've back linked this issue on the databricks website to this item as well.https://github.co...
- 1 kudos
- 917 Views
- 0 replies
- 1 kudos
How to Restrict Databricks Account based on IP/Subnet? To Avoid Security Risks related to accessing Databricks Environments to your employees, we have...
How to Restrict Databricks Account based on IP/Subnet?To Avoid Security Risks related to accessing Databricks Environments to your employees, we have a config that was published by databricks.we can Add IP/Subnets in IP Access List.when you enable fe...
- 917 Views
- 0 replies
- 1 kudos
- 6869 Views
- 4 replies
- 2 kudos
Resolved! Databricks fs ls does not work
I have configured the databricks CLI using token generated in Azure databricks configure --aad-tokenConfigure is success. When I used the command "databricks fs ls" I am running into an error as shown belowError: AttributeError: type object 'Retry' ...
- 6869 Views
- 4 replies
- 2 kudos
- 2 kudos
It seems the Databricks cli needs and update. This issue is discussed on StackOverflow. For the time being, installing an older version of `urllib3` should solve the issue. The method was removed since 2.X, the latest 1.X version is 1.26.15. You can ...
- 2 kudos
- 2022 Views
- 1 replies
- 4 kudos
spark 3.4 and databricks 13 introduce two new types of timestamps for handling time zone information:- TIMESTAMP WITH LOCAL TIME ZONE: This type assum...
spark 3.4 and databricks 13 introduce two new types of timestamps for handling time zone information:- TIMESTAMP WITH LOCAL TIME ZONE: This type assumes that the input data is in the session's local time zone and converts it to UTC before processing....
- 2022 Views
- 1 replies
- 4 kudos
- 4 kudos
This is helpful! Timestamps are always the reason to mess up the business logic as we know.
- 4 kudos
- 12073 Views
- 6 replies
- 5 kudos
Resolved! Clusters stuck on pending indefinitely (community edition)
From yesterday, suddenly clusters do not start and are in the pending state indefinitely (more than 30 minutes). From a previous post, I tried to add 443 port to the firewall but it doesn't help. In the clusters page, the message says: Finding instan...
- 12073 Views
- 6 replies
- 5 kudos
- 5047 Views
- 8 replies
- 0 kudos
I have successfully passed the test after completion of the course I haven't received any badge from your side as promised. I have been provided with a certificate which looks *****. Please provide me with the badge. My cetificate ID is ID: E-03DK31
I tried log in to > https://credentials.databricks.com/ with a registered email address and not able to find any Badge there. I need badge for the test i taken and passed, do needful.
- 5047 Views
- 8 replies
- 0 kudos
- 0 kudos
Hello!The same situation is valid for me. I couldn't reach my badge from the following link. Could you help me too? @Vidula Khanna​ https://customer-academy.databricks.com/legacy/lms/index.php%3Fr%3DmyActivities/index%26tab%3Dbadges
- 0 kudos
- 1176 Views
- 0 replies
- 9 kudos
New unified Databricks navigation - New Release to Public preview check it out : Databricks release a new navigation experience to public preview.  Th...
New unified Databricks navigation - New Release to Public preview check it out :Databricks release a new navigation experience to public preview. The goal is to reduce clicks and context switches required to complete tasks. The new experience include...
- 1176 Views
- 0 replies
- 9 kudos
- 1248 Views
- 0 replies
- 9 kudos
Databricks MarketplaceDatabricks Marketplace, an open forum for exchanging data products. Databricks Marketplace takes advantage of Delta Sharing to g...
Databricks MarketplaceDatabricks Marketplace, an open forum for exchanging data products. Databricks Marketplace takes advantage of Delta Sharing to give data providers the tools to share data products securely and data consumers the power to explore...
- 1248 Views
- 0 replies
- 9 kudos
- 1365 Views
- 1 replies
- 1 kudos
My cluster shows time to time this message, what is the permanent solution
MessageCluster terminated. Reason: Spark Image Download FailureHelpFailed to set up spark container due to an image download failure: Exception when downloading spark image: Stdout: /usr/local/bin/fastar /usr/local/bin/fastarIhave tried enabling the ...
- 1365 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi, This error can be surfaced due to various reasons, such as networking , slowness etc.. Is there any other significant errors showing up in the driver logs? Is this for a single cluster? Is this a new workspace? You can start with checking on the ...
- 1 kudos
| User | Count |
|---|---|
| 1629 | |
| 790 | |
| 511 | |
| 349 | |
| 287 |
