User16763506477
Databricks Employee
Databricks Employee

Hi @Roberto Baldrez​ , you will need to add the below configs to the cluster

spark.sql.streaming.metricsEnabled true
*.sink.servlet.class org.apache.spark.metrics.sink.MetricsServlet
*.sink.servlet.path /metrics/json
master.sink.servlet.path /metrics/master/json
applications.sink.servlet.path /metrics/applications/json

URL will change to "http://<driverIP>:<port>/metrics/json/" the one you mentioned is for DStream application

note: This gives limited streaming metrics. If you need all metrics you will need to add metrics sink to the cluster.

More info

View solution in original post