Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
We are currently using the event log to gain some visibility to what spark is doing under the hood but we frequently find that rollover logs are missing and we are left with a partial set. Is there a reason for this or a bug being tracked? Is there a setting we can configure to turn off rollover or some other setting we might be missing?
โ09-10-202407:24 AM - edited โ09-10-202407:35 AM
I'm also having this issue ๐ , it seems to occur much more frequently on azure than aws, but even on aws jobs sometimes some of the logs will be missing
Connect with Databricks Users in Your Area
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
