Thanks @NandiniN !

After some further investigation, we discovered that the SocketTimeouts do not function inside container most probably due to this reported docker case (Socket timeout not working only on docker)

Of course that 

TemporarilyUnavailableRetry

could still respond in scenarios where the SQL Warehouse is down and the startup will take significant time.

Serverless warehouse is naturally another choice, indeed.