Hi @anastassia_kor1,

For CPU-only training, TrainingArguments has a no_cuda flag that should be set.

For transformers==4.26.1 (MLR 13.0) and transformers==4.28.1 (MLR 13.1), there's an additional xpu_backend argument that needs to be set as well. Try using:

training_args = TrainingArguments(output_dir="outputs", no_cuda=True, xpu_backend="gloo")

 For transformers==4.29.2 (MLR 13.2), try using:

training_args = TrainingArguments(output_dir="outputs", no_cuda=True)

 It may be necessary to restart the cluster in order for this argument to take effect.