REST API
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-30-2023 07:53 PM
Creating an application to capture cluster metrics and sending HTTP REST request to the Spark History Server's API endpoint to retrieve a list of applications. This request doesn't generate logs in the Spark History Server's log files. The Spark History Server logs primarily capture information related to the Spark applications it serves and its own internal operation, not external API requests.
Ref : https://spark.apache.org/docs/latest/monitoring.html#rest-api
1) Sample REST request using command.
curl http://<hostname>:18080/api/v1/applications/
2) We should get the expected JSON response.
3) Check the History server logs /opt/mapr/spark/spark-2.4.8/logs/spark-mapr-org.apache.spark.deploy.history.HistoryServer*.out and we do NOT see any logging statements for the request #1.
Where does the REST api requests get's logged ?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-31-2023 08:34 AM
Do you have any examples how it can be logged on client side ? Enabling verbose logging on curl request dosen't help much.
e.g) curl -v http://<hostname>:18080/api/v1/applications/