TorchTitan
1 article in the last 30 days.
-
Training a 1T-Parameter MoE on EU public infrastructure with PyTorch and TorchTitan
The EUROPA consortium pretrained a 951.9B-parameter MoE on the JUPITER supercomputer with TorchTitan. Throughput rose from 90 to about 1,300 tokens/s per GPU on 384 GH200 GPUs.