Hi @ccsalt ,

This is a known limitation. Log rotation (renaming to log4j-YYYY-MM-DD-HH.log.gz) only happens on the hour boundary. The active log file log4j-active.log has always the same name and is overwritten if a cluster restart happens within one hour. 

As a workaround:

• Use a cluster-scoped init script to rename active log file before the cluster starts;
• Or switch to a job cluster, if possible. Each run will write its logs to a separate folder.

Hope it helps.

Best regards,