topic Re: SSL Error While Setting up databricks cli or installing library in Data Engineering

SSL Error While Setting up databricks cli or installing library

Rahul_Samant — Fri, 24 Jun 2022 11:58:28 GMT

How to fix below SSL error while setting up databricks Cli or installing library in cluster

Library installation attempted on the driver node of cluster *** and failed.

Please refer to the following error message to fix the library or contact Databricks support.

Error Code: DRIVER_LIBRARY_INSTALLATION_FAILURE. Error Message: org.apache.spark.SparkException:

Process List(/databricks/python/bin/pip, install, openpyxl, --disable-pip-version-check)

exited with code 1. WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None))

after connection broken by 'SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)'))':

/simple/openpyxl/

databricks cli also giving below error

Error: SSLError: HTTPSConnectionPool(host='***-XXXX.***.azuredatabricks.net', port=443):

Max retries exceeded with url: /api/2.0/clusters/list

(Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)')))

Re: SSL Error While Setting up databricks cli or installing library

Vivian_Wilfred — Fri, 24 Jun 2022 13:23:27 GMT

Hi @Rahul Samant , have you configured a proxy for your workspace? If yes, please make sure you have whitelisted all the necessary hosts. Also please try to upgrade your DBR to a newer version as we have many bug fixes deployed in newer DBR versions.

Re: SSL Error While Setting up databricks cli or installing library

Megan05 — Fri, 24 Jun 2022 20:57:04 GMT

I was getting an SSL when trying to set up secrets using the Databricks CLI. To fix the CLI SSL error I went to %USERPROFILE%\.databrickscfg (~/.databrickscfg on Unix, Linux, or macOS) from the file explorer on my local machine and added the insecure = True argument to the program.

Re: SSL Error While Setting up databricks cli or installing library

Rahul_Samant — Mon, 27 Jun 2022 06:32:00 GMT

Hi @Vivian Wilfred , i have created Databricks in our VNET using dedicated subnet and using 10.4 LTS , installing the cli in the vm within the same vnet so whitelisting shouldn't be issue right ?

Re: SSL Error While Setting up databricks cli or installing library

Rahul_Samant — Mon, 27 Jun 2022 06:33:20 GMT

HI @Megan Finley ,

I tried that workaround but it seems to be not working, also we can do tht in dev env but in other env this workaround will not be allowed

Re: SSL Error While Setting up databricks cli or installing library

Vivian_Wilfred — Wed, 29 Jun 2022 21:31:02 GMT

Hi @Rahul Samant the error message that you see is actually coming from the cluster when it tries to make a connection to the pip repro to install the package. Do you think there is a proxy in the connection between cluster (VM) and pip repo?

CLI just triggers the library installation on the cluster so the source of the CLI should matter in this case.

Re: SSL Error While Setting up databricks cli or installing library

Rahul_Samant — Thu, 30 Jun 2022 10:27:40 GMT

Thanks @Vivian Wilfred Issue of Library installation is resolved in the cluster after pointing to internal repository but CLI on VM is still failing while calling any cli commands with above error . so how can we make the cli to install the library using internal private repo ?

Re: SSL Error While Setting up databricks cli or installing library

Vivian_Wilfred — Thu, 30 Jun 2022 20:10:21 GMT

@Rahul Samant , can you paste the CLI command that is failing? In your case, if the cluster picks up libraries from the internal repository then it should not matter if triggered from CLI or UI, the installation should work. Please make sure the cluster consistently points to the internal repository.

Re: SSL Error While Setting up databricks cli or installing library

Rahul_Samant — Fri, 01 Jul 2022 06:23:16 GMT

@Vivian Wilfred i am just listing down the clusters to test the connectivity

i have also explicity installed the databricks-cli==0.17.0 in the cluster using private repository

databricks clusters list

Error: SSLError: HTTPSConnectionPool(host='adb-XXXXX.X.azuredatabricks.net', port=443): Max retries exceeded with url: /api/2.0/clusters/list (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)')))

how can we make sure the cluster consistently points to internal repository , as of now i see only this option while installing to provide the repo name but no option to persist the repo name for reuse ?

Re: SSL Error While Setting up databricks cli or installing library

Rahul_Samant — Wed, 20 Jul 2022 14:30:21 GMT

HI @Vivian Wilfred . any advice on this? am not able to use databricks cli due to ssl error ?