Tauon

Tauon brings faster training and lower loss to AI optimization

A new optimizer called Tauon is turning heads in the r/MachineLearning community, and for good reason. It promises faster training and lower loss by rethinking how we handle polynomials and orthogonalization, building on ideas from Muon but streamlining the process. The initial results are compelling: Tauon hit a lower validation loss than both Muon and AdamW, all while running nearly as fast as the baseline. It's a small-scale test, but the stability and compute efficiency suggest real potential.