Error Code: 3206 - Processed HTTP request failed.

[Atul_Sharan]

The ADF(Azur Data Factory) pipelines jobs executing several Databricks Notebook activities in parallel have been failing regularly with the following error "Error Code: 3206 - Processed HTTP request failed." The issue gets resolved on its own upon restarting the job. This issue started popping up from last 3 months.

The dev team have been adding new notebooks for parallel execution in the job quite frequently so I am not sure if the number databricks activities running in parallel causing this issue or something else. It would be very helpful if someone has any insights about how to manage this problem which has been happening quite regularly now.

Thanks !

[User15813097110 · Accepted solution]

Hi Atul,

You are correct. Such errors generally denote that service you are communicating is under heavy load. It is also possible that few of those requests are being throttled.

Please file a support case if this becomes too frequent. We will look closely into the exact scenario and identify a potential solution.

Best Regards,

Prakash

[Hubert-Dudek]

yes it is heavy load as you mention parallel runs of notebooks on one cluster so it is quite common issue. Easiest way is to decrease parallelism (use two clusters, don't start notebooks at the same time, driver could be bigger than executors nodes) than analyze your vitual machines (cpu usage, i/o, memory) and optimize machines than analyze your code (partitions, shuffles, data skews etc.)

[willjoe]

Method 1 - Close Conflicting Programs

down vote

When you get a runtime error, keep in mind that it is happening due to programs that are conflicting with each other. The first thing you can do to resolve the problem is to stop these conflicting programs.

Open Task Manager by clicking Ctrl-Alt-Del at the same time. This will let you see the list of programs currently running.

Go to the Processes tab and stop the programs one by one by highlighting each program and clicking the End Process button.

You will need to observe if the error message will reoccur each time you stop a process.

Once you get to identify which program is causing the error, you may go ahead with the next troubleshooting step, reinstalling the application.

Method 2 - Update / Reinstall Conflicting Programs

Using Control Panel

For Windows 7, click the Start Button, then click the Control panel, then Uninstall a program

For Windows 8, click the Start Button, then scroll down and click More Settings, then click Control panel > Uninstall a program.

For Windows 10, just type Control Panel on the search box and click the result, then click Uninstall a program

Once inside Programs and Features, click the problem program and click Update or Uninstall.

If you chose to update, then you will just need to follow the prompt to complete the process, however, if you chose to Uninstall, you will follow the prompt to uninstall and then re-download or use the application's installation disk to reinstall the program.

Using Other Methods

For Windows 7, you may find the list of all installed programs when you click Start and scroll your mouse over the list that appears on the tab. You may see on that list utility for uninstalling the program. You may go ahead and uninstall using the utilities available in this tab.

For Windows 10, you may click Start, then Settings, then choose Apps.

Scroll down to see the list of Apps and features installed on your computer.

Click the Program which is causing the runtime error, then you may choose to uninstall or click Advanced options to reset the application.

Regards,

Will