Our new paper on distributed optimization with strong optimization and DP guarantees is out! We introduce Clip21-SGD2M, a method featuring a double momentum mechanism—one for managing stochastic noise and another for averaging DP noise. We establish optimal convergence guarantees in both deterministic and stochastic settings, along with a near-optimal privacy-utility tradeoff in the DP framework. Finally, our method demonstrates competitive performance in practice efficiently handling the noise in training neural networks.

1️⃣ The paper Unbiased and Sign Compression in Distributed Learning: Comparing Noise Resilience via SDEs (will be available online soon), joint work with Enea Monzio Compagnoni, Frank Norbert Proske, and Aurelien Lucchi got accepted to AISTATS 2025 as Oral!
2️⃣ Two papers Towards Faster Decentralized Stochastic Optimization with Communication Compression, joint work with Yuan Gao and Sebastian Stich, and Adaptive Methods through the Lens of SDEs: Theoretical Insights on the Role of Noise, joint work with Enea Monzio Compagnoni, Tianlin Liu, Frank Norbert Proske, Antonio Orvieto, Aurelien Lucchi, got accepted to ICLR.
3️⃣ And finally, the paper Partiall Personalized Federated Learning: Breaking the Curse of Data Heterogeneity, joint work with Konstantin Mishchenko, Eduard Gorbunov, Samuel Horváth, got accepted to TMLR.

I presented our recent work on loss landscape characterization during NeurIPS@Paris workhsop poster session. My travel was supported by the organizers of the workshop. In this work, we propose a class of functions that better captures the properties of loss landscape of neural networks.

I am happy to announce that I have one paper accepted to NeurIPS 2024 on a novel class of functions that neural networks represent. This class contains functions with saddles and local minima in contrast to previous works. This is a joint work with Niccolò Ajroldi , Antonio Orvieto, and Aurelien Lucchi. It is now available on arXiv.

New paper on decentralized training with contractive compressed communication. We achieve near-optimal convergence guarantees using Error Feedback and momentum tracking.