We found out we were regeneratig the symlink manifest for all the partitions on this case. And for some reason it was executed twice, at start and end of the job.

delta_table.generate('symlink_format_manifest')

We configured the table with:

ALTER TABLE delta.`<path-to-delta-table>` SET TBLPROPERTIES(delta.compatibility.symlinkFormatManifest.enabled=true)

and removed the symlink creation on the job and all worked great from 2hs to 30 min.

Thanks!