Databricks to snowflake data load

SuMiT1
New Contributor III

Hi Team,

 

I’m trying to load data from Databricks into Snowflake using the Snowflake Spark connector. I’m using a generic username and password, but I’m unable to log in using these credentials directly. In the Snowflake UI, I can only log in through “Sign in with Azure AD.”

 

Below is the code snippet I’m using:

 

sfOptions = {

    "sfURL": "XD1.privatelink.snowflakecomputing.com",

    "sfDatabase": "Z_AICS",

    "sfSchema": "ASK_SMG",

    "sfWarehouse": "WAGT_ETL",

    "sfUser": "sys_aski@xyz.com",

    "sfPassword": "Qtr3tdint$",

    "sfRole": "AZURE_DATA_PLATFORMS"

}

 

selected_df.write \

    .format("snowflake") \

    .options(**sfOptions) \

    .option("dbtable", "DIM_TOPIC") \

    .mode("overwrite") \

    .save()

 

However, I’m getting the following error:

 

> Incorrect username or password was specified.

 

 

 

Could anyone please suggest how to authenticate and connect Snowflake from Databricks when Azure AD SSO is required for login?

nayan_wylde
Esteemed Contributor II

@SuMiT1  The recommended method to connect to snowflake from databricks is OAuth with Client Credentials Flow.

This method uses a registered Azure AD application to obtain an OAuth token without user interaction.

Steps:

  1. Register an app in Azure AD and configure it for Snowflake OAuth integration.
  2. Create a security integration in Snowflake using the client credentials.
  3. Use the token in your Spark job.

Here is the full guide.

https://community.snowflake.com/s/article/How-To-Connect-To-Snowflake-From-Azure-Databricks-Using-OA...

Other option is  OAuth with User Credentials Flow (Interactive Login)

This method uses a user's Azure AD credentials to obtain a token. It's less ideal for automation but works for interactive sessions.

https://snowforce.my.site.com/s/article/How-To-Connect-To-Snowflake-From-Azure-Databricks-Using-OAut...