Hello all,

I am suddenly getting an HTTP 502 and DRIVER_LIBRARY_INSTALLATION_FAILURE error during the Python library installation when the cluster gets initialized. I have around 10 Python packages out of which 2-3, packages always failed to install and packages. The confusing part is, these erroneous packages get changed each time I restart my cluster. Here are the primary details:

Platform: Microsoft Azure

Databricks Runtime Version: 12.2 LTS ML

Node Type: Standard F4

Can anyone please guide me on this? Thanks in advance!