Can I use OSS Spark History Server to view the EventLogs

[brickster_2018]

Is it possible to run the OSS SPark history server and view the spark event logs.

[brickster_2018 · Accepted solution]

Yes, it's possible. The OSS Spark history server can read the Spark event logs generated on a Databricks cluster.

Using Cluster log delivery, the SPark logs can be written to any arbitrary location. Event logs can be copied from there to the storage directory pointed by the OSS Spark History server.