Beyond Market Intelligence/Expert Parallelism

Expert Parallelism

Expert Parallelism at Beyond Market Intelligence is a file of one story. The newest of them: “Training a trillion-parameter AI on the GPUs powering Europe's fastest supercomputer”. 384 GPUs on Europe's fastest supercomputer, training a trillion-parameter model at 1,300 tokens per second per GPU, and the team still spent 68 seconds out of every 84 waiting on the network. Acme AI is the next-generation, AI-powered spreadsheet platform built to replace Excel and redefine how analysts, data scientists, and enterprise teams work… The list below is every Expert Parallelism story on Beyond Market Intelligence, newest first.

Machine Learning

Training a trillion-parameter AI on the GPUs powering Europe's fastest supercomputer

384 GPUs on Europe's fastest supercomputer, training a trillion-parameter model at 1,300 tokens per second per GPU, and the team still spent 68 seconds out of every 84 waiting on the network. That's the humbling starting point they turned into a 14X improvement. The real achievement here isn't just the scale, it's the upstream contributions. Every lesson, from FlexShard to FP8 all-gather, went back to PyTorch and TorchTitan so the next person doesn't have to rediscover them.