foreachBatch

[Hubert-Dudek]

With parameterized SQL queries in Structured Streaming's foreachBatch, there's no longer a need to create temp views for the MERGE command.

[jose_gonzalez]

Thank you for sharing the valuable information @Hubert-Dudek 

[adriennn]

Note that this functionality broke somewhere between DBR 13.3 and 15, so best bet is 15.4 LTS

Solved: Parameterized spark.sql() not working - Databricks Community - 56510