Need help to analyze databricks logs for a long-running job.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-06-2022 08:14 AM
Hi Team,
We have a job it completes in 3 minutes in one Databricks cluster, if we run the same job in another databricks cluster it is taking 3 hours to complete.
I am quite new to Databricks and need your guidance on how to find out where databricks spend a long-time during execution job. Any specific things I can check in Executor/driver logs?
Regards,
Rajesh.
- Labels:
-
Databricks Cluster
-
Databricks Logs
-
Logs
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-06-2022 09:54 PM
Hi Kaniz,
Sure, thanks for the response.
Regards,
Rajesh.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-10-2022 10:08 PM
Hi @Kaniz Fatma
I am able to extract the logs, but I need some pointer where the databricks spend a longer time during job execution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-07-2024 12:53 PM
Hi @Retired_mod ,
I am saving logs of my databricks Job Compute From ADF, How can i open those files that present in dbfs location.
