modular addition

Population Based Training reveals hidden escape routes from a dead zone in grokking

Population Based Training can pull a model out of a grokking dead zone, but it's a rescue tool, not a speed-up. In this test, PBT escaped a fixed setting where nothing groks, yet it landed slower than a plain grid search that already held the answer. The winning schedule mattered more than weight copying, and random search still won on speed. It's a useful lifeline for bad ranges, with caveats. For deeper coverage, our related piece on spline-based regression offers another angle on model efficiency.