connection from databricks to snowflake using OKTA

[ymt]

Hi team,

This is how I connect to Snowflake from Jupyter Notebook:

import snowflake.connector

snowflake_connection = snowflake.connector.connect(
    authenticator='externalbrowser',
    user='U1',
    account='company1.us-east-1',
    database='db1',
    schema='s1',
    warehouse='WH1'
)

cur = snowflake_connection.cursor()

cur.execute("select col1 from tab1;")

(it redirects me to SSO / OKTA auth page and then works fine)

This is what I get when I try to do the same in my Databricks Notebook:

ImportError: cannot import name 'NamedTuple' from 'typing_extensions' (/databricks/python/lib/python3.9/site-packages/typing_extensions.py)

The error happens on my "snowflake_connection = snowflake.connector.connect" line.

Wait, what?

Ok, this is how I tried to solve it using the very first cell in my Databricks Notebook:

%sh
pip install --upgrade pip
pip install snowflake-connector-python
pip install --upgrade typing_extensions

It worked without error, but didn't actually help.

I've seen other topics regarding Databricks to Snowflake connection here, they suggest to use secrets with Snowflake username / password pair, but I don't have any Snowflake password, I authenticate using SSO / OKTA.

Please advise.

[Kaniz_Fatma]

Hi @ymt, It seems you’ve encountered an issue while connecting to Snowflake from your Databricks Notebook.

The error message you received is:

ImportError: cannot import name 'NamedTuple' from 'typing_extensions' (/databricks/python/lib/python3.9/site-packages/typing_extensions.py)

This error occurs when the NamedTuple class cannot be imported from the typing_extensions module. The root cause lies in the compatibility between the Snowflake connector and the version of typing_extensions being used.

Here are some steps you can take to address this issue:

1. Check Python Version and Environment:

   • Ensure that you are using a compatible Python version. The Snowflake connector may have specific requirements.
   • Verify that your Databricks environment is using the correct Python interpreter.

2. Update typing_extensions:

   • The error suggests that the NamedTuple class is missing. This class was added in typing_extensions version 4.3.0.
   • Upgrade your typing_extensions package to a version equal to or greater than 4.3.0. You can do this using the following command:

     %sh
     pip install --upgrade typing_extensions
     

   • After upgrading, try connecting to Snowflake again.

3. Check for Module Name Conflicts:

   • Sometimes, conflicts arise due to module names. Ensure that you don’t have any custom modules or files named snowflake.py or typing_extensions.py in your Databricks workspace. Such conflicts can lead to unexpected behaviour during imports¹.

4. Use Secrets with SSO/OKTA:

   • Since you authenticate using SSO/OKTA, consider using secrets to securely store your Snowflake username and password pair.
   • Databricks provides a way to manage secrets, allowing you to avoid hardcoding credentials in your code.
   • You can create a secret scope and store your Snowflake credentials there. Then, reference the secret in your Databricks Notebook.

Remember that the Snowflake connector’s compatibility with Python libraries can impact its functionality. By following these steps, you should be able to resolve the issue and establish a successful connection to Snowflake from your Databricks environment.

If you encounter any further challenges, feel free to ask for additional assistance! 😊