foreachBatch

Hubert-Dudek
Databricks MVP

‎10-11-2023 01:26 PM

With parameterized SQL queries in Structured Streaming's foreachBatch, there's no longer a need to create temp views for the MERGE command.

structured1.png


My blog: https://databrickster.medium.com/

jose_gonzalez
Databricks Employee
Databricks Employee

‎10-12-2023 03:49 PM

Thank you for sharing the valuable information @Hubert-Dudek 

adriennn
Valued Contributor

‎09-18-2024 08:52 AM

Note that this functionality broke somewhere between DBR 13.3 and 15, so best bet is 15.4 LTS

Solved: Parameterized spark.sql() not working - Databricks Community - 56510

0 Kudos