Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-10-2023 12:32 AM
Does Spark support multi statement writes to Snowflake in a single session? To elaborate, I have a requirement where I need to do
- A selective deletion of data from a Snowflake table and
- Insert records to Snowflake table ( Ranges from around 1 M rows)
I want to wrap these both into a single transaction so that I could get Consistency here. I can't use overwrite mode as it will overwrite my entire table in Snowflake. I have seen runQuery of sfutils support multiStatements but it is more to handle data within Snowflake and not from Databricks to Snowflake.
Thanks,
Vignesh