Issue Importing transformers Library on Databricks

Deniz_Bilgin
New Contributor

‎06-26-2024 05:04 AM

I'm experiencing an issue when trying to import the "transformers" library in a Databricks notebook. The import statement causes the notebook to hang indefinitely without any error messages. The library works perfectly on my local machine using Anaconda.

Does anyone experience the same problem?

0 Kudos

db_pv1504
New Contributor II

‎09-24-2025 08:07 AM

I am also facing this issue with 17.2/17.1 ml runtime with no changes to the environment. Only options I found was upgrading transformers to the latest version or revert back to 16.4 lts

0 Kudos

nayan_wylde
Esteemed Contributor II

‎09-24-2025 08:16 AM

@Deniz_Bilgin yeah some packages are not compatible with runtime.

  •     Use a Stable Version
         Try installing a known working version:
%pip install transformers==4.41.2

  • Dependency Issues:

    Conflicts with preinstalled libraries like urllib3 (e.g., version 2.1.0) in Databricks Runtime 13.3 can break transformers.

  • Add retry logic to handle intermittent import failures:
import time
max_attempts = 3
attempt = 0
while attempt < max_attempts:
    try:
        import transformers
        break
    except ImportError as e:
        attempt += 1
        print(f"Attempt {attempt} failed. Retrying...")
        time.sleep(10)

 

0 Kudos