Hi @NishantTiwari,
I see you have already upgraded to DBR 14.3+ but are still hitting the same SSL errors. That makes sense, and here is why: the two errors you are seeing point to the 3rd party server using weak or outdated SSL certificates, not an issue on your Databricks cluster itself.
CA_MD_TOO_WEAK - The server's CA certificate uses a weak message digest (e.g., MD5 or short SHA-1)
EE_KEY_TOO_SMALL - The server's end-entity certificate uses a key that is too short (e.g., 1024-bit RSA)
Newer Databricks Runtimes ship with updated versions of OpenSSL that enforce stricter security defaults. So upgrading the runtime actually makes OpenSSL MORE strict, which is why you continue to see these errors even on 14.3+.
THE RECOMMENDED LONG-TERM FIX
The best solution is to ask the 3rd party portal to update their TLS certificates to use modern standards (at minimum TLS 1.2 with 2048-bit RSA keys and SHA-256 or stronger). This is the correct fix because their certificates do not meet current security standards.
A WORKAROUND USING AN INIT SCRIPT
If you cannot get the 3rd party to update their certificates right away, you can temporarily lower the OpenSSL security level on your cluster using an init script. This allows the connection to succeed while the 3rd party works on upgrading their certificates.
Step 1: Create the init script file. In your Databricks workspace, create a new file (for example at /Workspace/Users/your-email/init-scripts/lower-ssl-security.sh) with this content:
#!/bin/bash
# Temporarily lower OpenSSL security level to allow weak certificates
# from legacy third-party servers.
OPENSSL_CONF_FILE="/etc/ssl/openssl_custom.cnf"
cat > "$OPENSSL_CONF_FILE" << 'EOF'
openssl_conf = default_conf
[default_conf]
ssl_conf = ssl_sect
[ssl_sect]
system_default = system_default_sect
[system_default_sect]
CipherString = DEFAULT:@SECLEVEL=1
EOF
echo "export OPENSSL_CONF=$OPENSSL_CONF_FILE" >> /etc/environment
echo "export OPENSSL_CONF=$OPENSSL_CONF_FILE" >> /databricks/spark/conf/spark-env.sh
Step 2: Attach the init script to your cluster.
1. Go to your cluster configuration page
2. Enable the Advanced toggle
3. Click the Init Scripts tab
4. Select "Workspace" as the source
5. Enter the path to your script (e.g., /Workspace/Users/your-email/init-scripts/lower-ssl-security.sh)
6. Click Add, then restart the cluster
AN ALTERNATIVE PYTHON-LEVEL WORKAROUND
If you only need this for a specific notebook or job step rather than the whole cluster, you can also set the environment variable directly in your Python code before making the HTTPS call:
import os
import ssl
import urllib3
# Lower the security level for this session only
os.environ['OPENSSL_CONF'] = '/dev/null'
# Create a custom SSL context with lower security
ctx = ssl.create_default_context()
ctx.set_ciphers('DEFAULT:@SECLEVEL=1')
# If using requests library:
import requests
from requests.adapters import HTTPAdapter
class SSLAdapter(HTTPAdapter):
def init_poolmanager(self, *args, kwargs):
ctx = ssl.create_default_context()
ctx.set_ciphers('DEFAULT:@SECLEVEL=1')
kwargs['ssl_context'] = ctx
return super().init_poolmanager(*args, kwargs)
session = requests.Session()
session.mount('https://', SSLAdapter())
response = session.get('https://your-third-party-url.com/download')
IMPORTANT NOTES
- Lowering the SSL security level does reduce the security of those connections, so only apply this to the specific cluster or session that needs it.
- This should be treated as a temporary measure while the 3rd party upgrades their certificates.
- Make sure you are running DBR 13.3 LTS or later since DBR 7.3 has reached end-of-life as of February 2026 and will no longer launch clusters.
For reference on configuring init scripts:
https://docs.databricks.com/aws/en/init-scripts/cluster-scoped.html
For the Databricks Runtime support schedule:
https://docs.databricks.com/aws/en/archive/runtime-release-notes/
Hope this helps get your NDS download step working again!
* This reply used an agent system I built to research and draft this response based on the wide set of documentation I have available and previous memory. I personally review the draft for any obvious issues and for monitoring system reliability and update it when I detect any drift, but there is still a small chance that something is inaccurate, especially if you are experimenting with brand new features.
