Resources

LECTURE NOTES, REVIEWS AND BLOG POSTS

Replica Method for the Machine Learning Theorist - Part 1, by Blake Bordelon, Haozhe Shan, Abdul Canatar, Boaz Barak, Cengiz Pehlevan

Replica Method for the Machine Learning Theorist - Part 2, by Blake Bordelon, Haozhe Shan, Abdul Canatar, Boaz Barak, Cengiz Pehlevan

Lecture notes on the replica method for Wishart matrix eigenvalues, by Jacob Zavatone-Veth

A brief introduction to the neural network Gaussian process from the perspective of mean field theory, by Jacob Zavatone-Veth

Lecture Notes on Infinite-Width Limits of Neural Networks, Cengiz Pehlevan and Blake Bordelon, prepared for 2023 Princeton ML Theory Summer School

Infinite Limits of Neural Networks, Deeper Learning Blog, by Alex Atanasov, Blake Bordelon and Cengiz Pehlevan

A Dynamical Model of Neural Scaling Laws, Deeper Learning Blog, by Blake Bordelon, Alex Atanasov and Cengiz Pehlevan

Scaling and renormalization in high-dimensional regression, by Alex Atanasov, Jacob Zavatone-Veth, Cengiz Pehlevan

Disordered Dynamics in High Dimensions: Connections to Random Matrices and Machine Learning, by Blake Bordelon and Cengiz Pehlevan

Solvable Model of In-Context Learning Using Linear Attention, Deeper Learning Blog, by Mary Letey

Anytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averaging, Deeper Learning Blog, by Alexandru Meterez*, Pranav Ajit Nair*, Depen Morwani*, Cengiz Pehlevan, Sham Kakade

Jailbreak Scaling Laws for Large Language Models: Polynomial–Exponential Crossover, Deeper Learning Blog, by Indranil Halder, Annesya Banerjee, Cengiz Pehlevan
 

CODE

Group GitHub

VIDEOS

Physics of Learning Collaboration Webinar Series

KITP Program on Deep Learning from the Perspective of Physics and Neuroscience

Pehlevan Group Youtube Channel