DLT aggregation problem

Yash_542965
Databricks Partner

I'm utilizing SQL to perform aggregation operations within a gold layer of a DLT pipeline. However, I'm encountering an error when running the pipeline while attempting to return a data frame using spark.sql.

Could anyone please assist me with the SQL syntax required to apply a watermark within the SQL query? It would be greatly appreciated if someone could provide an example of the syntax for applying a watermark in SQL.

ERROR: "org.apache.spark.sql.AnalysisException: Append output mode not supported when there are streaming aggregations on streaming DataFrames/DataSets without watermark;"

lucasrocha
Databricks Employee
Databricks Employee

Hello @Yash_542965 , I hope this message finds you well.

Could you please share a sample of code you are using so that we can check it further?

Best regards,
Lucas Rocha