The ADF(Azur Data Factory) pipelines jobs executing several Databricks Notebook activities in parallel have been failing regularly with the following error "Error Code: 3206 - Processed HTTP request failed." The issue gets resolved on its own upon restarting the job. This issue started popping up from last 3 months.

The dev team have been adding new notebooks for parallel execution in the job quite frequently so I am not sure if the number databricks activities running in parallel causing this issue or something else. It would be very helpful if someone has any insights about how to manage this problem which has been happening quite regularly now.

Thanks !