Hi @Guillermo-HR
Yes — batch is usually the right fix here.
What’s happening is that your query is using event-time window aggregation in Structured Streaming with append output mode. In that mode, Spark only emits a window after it is sure the wind...
Hi @FAHADURREHMAN
Please find below my findings
Since you're using Auto Loader (cloudFiles) this behavior is expected.
By default, when you provide a path like:
s3://bucket/folder/
Spark recursively reads all subfolders.pathGlo...
Hi @dtb_usr
Thank you for sharing the error. I understand now. Can you raise a support ticket with the databricks team we need to dig deeper into the issue.
