01-05-2022 05:22 PM
Hi @Timothy Lin ,
I will recommend to not use spark.stop() or System.exit(0) in your code because it will explicitly stop the Spark context but the graceful shutdown and handshake with databricks' job service does not happen.
