Delta Live table getting error of garbage collection after running few days

Darian
New Contributor II

Hi, i am using delta live table in continuous mode for a real time streaming data pipeline. After running the pipeline like 2-3 days i am getting this garbage collection error:

Driver/10.15.0.73 paused the JVM process 68 seconds during the past 120 seconds (57.32%) because of GC. We observed 5 such issue(s) since 2024-04-29T15:41:29.874Z.

Darian_0-1714426883477.png

Here are my cluster setting:

Darian_1-1714426964675.png