brickster_2018
Databricks Employee
Databricks Employee

Yes, it's possible. The OSS Spark history server can read the Spark event logs generated on a Databricks cluster.

Using Cluster log delivery, the SPark logs can be written to any arbitrary location. Event logs can be copied from there to the storage directory pointed by the OSS Spark History server.

View solution in original post