🚨 New paper alert! 🚨
Our latest work has been accepted to NeurIPS! We demonstrate how adapting the NGN stepsize with momentum enhances optimization both theoretically and practically.
What’s inside?
📚 Theory: Convergence in convex and non-convex settings.
🔧 Cleaner analysis: We drop restrictive assumptions (no bounded gradients, no interpolation), improving over prior work on Stochastic Polyak stepsize.
🚀 Practice: NGN + momentum and NGN + Adam = way more robust to learning rate choices, from ResNets to large-scale LMs.
Special credit to my amazing coauthors: Niccolò Ajroldi, Antonio Orvieto, Aurelien Lucchi.

I am happy to announce that our recent that studies Safe-EF algorithm for distributed non-smooth optimization with constraints has been accepted to ICML 2025! Quick summary of our contributions:
⚙️ Safe-EF algorithm which provably converges in the federated non-smooth optimization with constraints;
🚨 Algorithms like Compressed SGD and EF21 fail to converge in such a setting;
🛠️ Lower bounds for first-order methods in the compressed non-smooth convex setting.
🔄 High probability analysis in the stochastic setting;
🤖 Extensive experiments, including distributed humanoid training, show that Safe-EF efficiently minimizes the objective while preserving constraint violations. This is a joint work with Yarden As and Ilyas Fatkhullin. Check out our paper for more details!

Our new paper on distributed optimization with strong optimization and DP guarantees is out! We introduce Clip21-SGD2M, a method featuring a double momentum mechanism—one for managing stochastic noise and another for averaging DP noise. We establish optimal convergence guarantees in both deterministic and stochastic settings, along with a near-optimal privacy-utility tradeoff in the DP framework. Finally, our method demonstrates competitive performance in practice efficiently handling the noise in training neural networks.

1️⃣ The paper Unbiased and Sign Compression in Distributed Learning: Comparing Noise Resilience via SDEs (will be available online soon), joint work with Enea Monzio Compagnoni, Frank Norbert Proske, and Aurelien Lucchi got accepted to AISTATS 2025 as Oral!
2️⃣ Two papers Towards Faster Decentralized Stochastic Optimization with Communication Compression, joint work with Yuan Gao and Sebastian Stich, and Adaptive Methods through the Lens of SDEs: Theoretical Insights on the Role of Noise, joint work with Enea Monzio Compagnoni, Tianlin Liu, Frank Norbert Proske, Antonio Orvieto, Aurelien Lucchi, got accepted to ICLR.
3️⃣ And finally, the paper Partiall Personalized Federated Learning: Breaking the Curse of Data Heterogeneity, joint work with Konstantin Mishchenko, Eduard Gorbunov, Samuel Horváth, got accepted to TMLR.

I presented our recent work on loss landscape characterization during NeurIPS@Paris workhsop poster session. My travel was supported by the organizers of the workshop. In this work, we propose a class of functions that better captures the properties of loss landscape of neural networks.