Hi,

Current approach reloads dim_df in every batch, which can be inefficient. To optimize, consider broadcasting dim_df if it's small or using a mapGroupsWithState function for stateful joins. Also, ensure that fact_df has sufficient watermarking to handle late data efficiently. Let me know if you need further optimization suggestions!

Regards,
Bryce June