I'm utilizing SQL to perform aggregation operations within a gold layer of a DLT pipeline. However, I'm encountering an error when running the pipeline while attempting to return a data frame using spark.sql.
Could anyone please assist me with the SQL syntax required to apply a watermark within the SQL query? It would be greatly appreciated if someone could provide an example of the syntax for applying a watermark in SQL.
ERROR: "org.apache.spark.sql.AnalysisException: Append output mode not supported when there are streaming aggregations on streaming DataFrames/DataSets without watermark;"
