ava.net.SocketTimeoutException - ReadTimeOut

agar08 — Fri, 28 Jul 2023 17:22:18 GMT

Databricks notebook is connecting ADLS Gen2 using Service principal authentication and the setup is working fine. The notebook is able to read/write files to ADLS Gen2. However, occasionally, we are seeing below error in the production environment:

java.net.SocketTimeoutException at java.net.SocketInputStream.socketRead

HttpRequestFailure: 0,err=,appendpos=,cid=b165d74b-8177-4e24-9226-c4ce42dda733,rid=,connMs=0,sendMs=0,recvMs=0,sent=0,recv=0,method=HEAD,url=https://zXXXXXX01.dfs.core.windows.net/clusters/user/cxxxxxy/rxx-inc-json/xx/x/7xxx4/xxx-xxxx-xxxx.json?upn=false&action=getStatus&timeout=90

------

Cluster Details:

------

"spark_version": "10.4.x-scala2.12",

"node_type_id": "Standard_D8s_v5",
"driver_node_type_id": "Standard_D16ds_v5"

---

Any help you can provide would be greatly appreciated.

Thanks

topic ava.net.SocketTimeoutException - ReadTimeOut in Data Engineering

ava.net.SocketTimeoutException - ReadTimeOut