Upgrading Drivers and Authentication Method for Snowflake Integration

[Vinil]

Hello Databricks Support Team,

I am reaching out to request assistance with upgrading the drivers and configuring authentication methods for our Snowflake–Databricks integration.

We would like to explore and implement one of the recommended secure authentication mechanisms as outlined in Snowflake documentation:

• Key Pair Authentication

• OAuth Authentication

• Programmatic Access Tokens

  Could you please provide guidance on:

  1. The driver upgrade process required to support these authentication methods.

  2. Recommended authentication approach for Snowflake connectivity from Databricks.

  3. Any configuration steps or best practices we should follow during this transition.

     We would appreciate it if you could also share any relevant Databricks documentation or implementation examples that would help streamline this upgrade.

     Thank you for your support, and we look forward to your guidance.

     Best regards,
     Vinil Kuragayala

[Khaja_Zaffer]

Hello @Vinil 

Greatings for the day!

For Snowflake connectivity from Databricks, OAuth is generally recommended as the most secure and scalable method, especially for production environments. You can find more details over here: https://docs.databricks.com/aws/en/query-federation/snowflake 

I have a video on the same for you: https://www.youtube.com/watch?v=Re5Kg64Tg50 

Here are documentation from snowflake for your referenc: 

You can build an OAuth integration to Snowflake ourselfs using Entra ID: https://community.snowflake.com/s/article/External-oAuth-Token-Generation-using-Azure-AD

For Downloading and Installing the Connector for spark:

https://docs.snowflake.com/en/user-guide/spark-connector-install#:~:text=OCSP%20Configuration.-,Down...

As this is very open question, I am open for the other solutions as well. 

Thank you. 

Note: Once you find any solution which aligned to the query, please do not forget to mark it as accepted solution.  

[Vinil]

Hi @Khaja_Zaffer ,

Thanks for the details but need help on implementing in OAUTH. Please provide your availability to set up call today.

[Khaja_Zaffer]

Hello @Vinil

Are you getting any errors while trying to create the connections.

Please let me know if you follow this documentation : https://learn.microsoft.com/en-us/azure/databricks/query-federation/snowflake-oauth-access-token 

[Vinil]

@Khaja_Zaffer ,

Currently we are using Azure key vault for Snowflake authentication and it is used more than 300 jobs in Databricks. Want to implement without impact on the production jobs. Please set up call ASAP today. 

[Khaja_Zaffer]

As you mentioned production jobs, I  think the better way is to cut a ticket from azure side, so that someone can assist you with the case. You have keyvault team and azure databricks team available if anytihng required. 300 jobs is big task mate. 

We can wait for any other contributor to share their expeirence. 

[Vinil]

@Khaja_Zaffer ,

Need assistance on upgrading Snowflake drivers on Cluster. We installed Snowflake package on cluster, How to upgrade Snowflake library? 

For Authentication, I will reach out to Azure team.

Thanks for the details.

[Khaja_Zaffer]

Hello @Vinil 

How are you doing today!

Apologies I was busy with my personal issues. 

Did you managed to work on the issue?