cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks to snowflake data load

SuMiT1
New Contributor III

Hi Team,

 

I’m trying to load data from Databricks into Snowflake using the Snowflake Spark connector. I’m using a generic username and password, but I’m unable to log in using these credentials directly. In the Snowflake UI, I can only log in through “Sign in with Azure AD.”

 

Below is the code snippet I’m using:

 

sfOptions = {

    "sfURL": "XD1.privatelink.snowflakecomputing.com",

    "sfDatabase": "Z_AICS",

    "sfSchema": "ASK_SMG",

    "sfWarehouse": "WAGT_ETL",

    "sfUser": "sys_aski@xyz.com",

    "sfPassword": "Qtr3tdint$",

    "sfRole": "AZURE_DATA_PLATFORMS"

}

 

selected_df.write \

    .format("snowflake") \

    .options(**sfOptions) \

    .option("dbtable", "DIM_TOPIC") \

    .mode("overwrite") \

    .save()

 

However, I’m getting the following error:

 

> Incorrect username or password was specified.

 

 

 

Could anyone please suggest how to authenticate and connect Snowflake from Databricks when Azure AD SSO is required for login?

1 REPLY 1

nayan_wylde
Esteemed Contributor

@SuMiT1  The recommended method to connect to snowflake from databricks is OAuth with Client Credentials Flow.

This method uses a registered Azure AD application to obtain an OAuth token without user interaction.

Steps:

  1. Register an app in Azure AD and configure it for Snowflake OAuth integration.
  2. Create a security integration in Snowflake using the client credentials.
  3. Use the token in your Spark job.

Here is the full guide.

https://community.snowflake.com/s/article/How-To-Connect-To-Snowflake-From-Azure-Databricks-Using-OA...

Other option is  OAuth with User Credentials Flow (Interactive Login)

This method uses a user's Azure AD credentials to obtain a token. It's less ideal for automation but works for interactive sessions.

https://snowforce.my.site.com/s/article/How-To-Connect-To-Snowflake-From-Azure-Databricks-Using-OAut...