Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-09-2024 08:43 AM
@dmytro yes, it's possible to monitor the consumer lag through the streaming query metrics. Every cluster that runs a spark structured streaming query will log the metrics for each streaming batch in the driver logs and Spark UI. More details at Monitoring Structured Streaming queries on Databricks.
Best regards,
Raphael Balogo
Sr. Technical Solutions Engineer
Databricks
Raphael Balogo
Sr. Technical Solutions Engineer
Databricks