Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-20-2021 04:23 PM
Hi @Roberto Baldrez , you will need to add the below configs to the cluster
spark.sql.streaming.metricsEnabled true
*.sink.servlet.class org.apache.spark.metrics.sink.MetricsServlet
*.sink.servlet.path /metrics/json
master.sink.servlet.path /metrics/master/json
applications.sink.servlet.path /metrics/applications/jsonURL will change to "http://<driverIP>:<port>/metrics/json/" the one you mentioned is for DStream application
note: This gives limited streaming metrics. If you need all metrics you will need to add metrics sink to the cluster.
More info