Delta Live table getting error of garbage collection after running few days

[Darian]

Hi, i am using delta live table in continuous mode for a real time streaming data pipeline. After running the pipeline like 2-3 days i am getting this garbage collection error:

Driver/10.15.0.73 paused the JVM process 68 seconds during the past 120 seconds (57.32%) because of GC. We observed 5 such issue(s) since 2024-04-29T15:41:29.874Z.

Here are my cluster setting:

[jose_gonzalez]

could you share the ganglia metrics  and how size/type is your driver?

[Darian]

Here are the metrics:

The size/type:

Thanks!