Anytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averaging
Publication information:
A. Meterez*, P. A. Nair*, D. Morwani*, C. Pehlevan, and S. Kakade,
“Anytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averaging”, High-dimensional Learning Dynamics (HiLD) Workshop at ICML (Oral), 2026.