How to Get spark history server logs using python REST API
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-22-2023 06:17 AM
Hello,
I am trying to load logs from spark job in remote location using Python Rest API.
I want to collect those logs for particular job runs using runID field, that log should contain Errors, Exceptions, and print details.
I have tried "/api/2.1/jobs/runs/get" and "/api/2.1/jobs/runs/get-output" but didnt get desired output logs.
basically I want logs from spark history server.
Let me know what method or call I can use in such case
Thanks in Advance
Parimal
1 REPLY 1
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-23-2023 11:14 PM
Hello,
How can I access endpoint (http URL) of spark history server for a particular cluster?
Thanks,
Parimal