Hi @NotCuriosAtAll ,

You have undersized cluster for your workload. This error is typical on driver node with that high cpu consumption. You can check below article (and related solution):
Job run fails with error message “Could not reach driver of cluster” - Databricks

If I were you I would just increase your compute. Your job sometimes can work beacuse each day the amount of data is bit different. But if you see nearly 100% of cpu and memory consumption each day then for sure your workload demands bigger compute (or optimization of your code)