topic Re: access cluster executor logs in Data Engineering

access cluster executor logs

mayur_05 — Fri, 20 Sep 2024 14:26:58 GMT

Hi Team,

I want to get realtime log for cluster executor and driver stderr/stdout log while performing data operations and save those log in catalog's volume

gchandra — Fri, 20 Sep 2024 15:28:09 GMT

Please enable this and it creates the log under /dbfs

periodically files from /dbfs to /volume

mayur_05 — Fri, 20 Sep 2024 15:47:44 GMT

is it possible for job run

gchandra — Fri, 20 Sep 2024 15:56:03 GMT

you can create it for Job Clusters compute too. The specific cluster log folder will be under /dbfs/cluster-logs (or whatever you change it to)