https://community.databricks.com/t5/machine-learning/what-s-torchdistributor/m-p/6252#M282 Fri, 07 Apr 2023 10:22:48 GMT youssefmrini 2023-04-07T10:22:48Z https://community.databricks.com/t5/machine-learning/what-s-torchdistributor/m-p/6252#M282 Fri, 07 Apr 2023 10:22:48 GMT https://community.databricks.com/t5/machine-learning/what-s-torchdistributor/m-p/6252#M282 youssefmrini 2023-04-07T10:22:48Z https://community.databricks.com/t5/machine-learning/what-s-torchdistributor/m-p/6253#M283 <P><A href="https://github.com/apache/spark/blob/master/python/pyspark/ml/torch/distributor.py" alt="https://github.com/apache/spark/blob/master/python/pyspark/ml/torch/distributor.py" target="_blank">TorchDistributor</A>&nbsp;is an open-source module in PySpark that helps users do distributed training with PyTorch on their Spark clusters, so it lets you launch PyTorch training jobs as Spark jobs.</P><P>With Databricks Runtime 13.0 ML and above, you can perform distributed training on PyTorch ML models using TorchDistributor. See&nbsp;<A href="https://docs.databricks.com/machine-learning/train-model/distributed-training/spark-pytorch-distributor.html" alt="https://docs.databricks.com/machine-learning/train-model/distributed-training/spark-pytorch-distributor.html" target="_blank">Distributed training with TorchDistributor</A> <A href="https://docs.databricks.com/machine-learning/train-model/distributed-training/spark-pytorch-distributor.html" target="test_blank">https://docs.databricks.com/machine-learning/train-model/distributed-training/spark-pytorch-distributor.html</A></P> Fri, 07 Apr 2023 10:23:19 GMT https://community.databricks.com/t5/machine-learning/what-s-torchdistributor/m-p/6253#M283 youssefmrini 2023-04-07T10:23:19Z