Hi @subhas_1729,
You can achieve this by leveraging Spark's monitoring and instrumentation APIs.
Spark provides metrics that can be accessed through the SparkListener interface as well as the REST API. The SparkListener interface allows you to receive notifications about events happening in your Spark application, such as job start/end and stage start/end. By implementing SparkListener and registering it with your SparkContext, you can collect metrics and other information that are typically displayed in the Spark UI
Implementing these approaches will allow your Spark program to access the required metrics and variables to design an informative dashboard similar to the Spark UI.
https://spark.apache.org/docs/latest/monitoring.html
