Hey @-werners- thanks for answering first, why then the metrics of cpu, mem utilisation we are getting in driver only and worker seems still, with less utilisation of any training, with torch distributor I think atleast that one worker should be in use, right ?

one more thing, are the databricks driver machine designed in such a way that makes it less optimal and performant for the model training and inference tasks. as databricks implies that the code should be in apache spark only ( keeping pytorch and pandas out of execution line).