cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Spark Logs inaccessible - from the UI and dbfs (GCS)

FurqanAmin
New Contributor II

‎01-29-2024 12:43 AM

We have a lot of jobs with spark-submit tasks, previously we were able to see the logs for the jobs. Now we are not able to see the logs in the DBX UI.
We created a test job for this 'test_job_2' in our workspace to test it out. When the job finishes executing, the logs for it are not accessible in the UI, even if we define a dbfs path of the Logging in the config of the job, the logs are still not accessible on the UI nor the dbfs location.
While the job is running, the logs are accessible, but as soon as it terminates the logs disappear.
0 Kudos
1 REPLY 1

Yeshwanth
Databricks Employee
Databricks Employee

‎01-31-2024 07:55 AM

@FurqanAmin Could you please attach a screenshot of this?

0 Kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Announcements