https://community.databricks.com/t5/data-engineering/databricks-job-cluster-logs/m-p/73971#M34689 <P>Hello, how can I enable Databricks to generate a separate spark log for each job run?</P><P>What parameters should I use in spark configuration?</P><P>&nbsp;</P> Fri, 14 Jun 2024 03:10:38 GMT Jackson1111 2024-06-14T03:10:38Z https://community.databricks.com/t5/data-engineering/databricks-job-cluster-logs/m-p/73971#M34689 <P>Hello, how can I enable Databricks to generate a separate spark log for each job run?</P><P>What parameters should I use in spark configuration?</P><P>&nbsp;</P> Fri, 14 Jun 2024 03:10:38 GMT https://community.databricks.com/t5/data-engineering/databricks-job-cluster-logs/m-p/73971#M34689 Jackson1111 2024-06-14T03:10:38Z https://community.databricks.com/t5/data-engineering/databricks-job-cluster-logs/m-p/74076#M34715 <P><a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/106242">@Jackson1111</a>&nbsp;- If you are talking about workflow jobs, you can try running using a job cluster to generate spark logs for a each of the workflow jobs.&nbsp;</P> <P>But, If this is of Spark Jobs within the Spark UI, you wanted to separate out the logs. This is a new feature. you can raise it is an idea -&nbsp;<A href="https://www.databricks.com/feedback" target="_blank">https://www.databricks.com/feedback</A>.&nbsp;</P> Fri, 14 Jun 2024 20:10:11 GMT https://community.databricks.com/t5/data-engineering/databricks-job-cluster-logs/m-p/74076#M34715 shan_chandra 2024-06-14T20:10:11Z