Hi @Jose Gonzalez​ , @Werner Stinckens​  @Kaniz Fatma​ ,

Thanks for your response .Appreciate a lot.

The issue was in the code, it was a python /panda code running on Spark. Due to this only driver node was being used. i did validate this by increasing the driver configuration. The next steps is to revisit the code and use RDD/dataframes so code has some parallel processing