בית הספר
להנדסת חשמל ומחשבים

Title: Modern Generative Models: From Generation to Uncertainty-Aware Inference Bio: Yi Zhang is a Postdoctoral Fellow in the Department of Computer Science and Applied Mathematics at the Weizmann Institute of Science, Israel. He received the B.E. degree in electronic engineering from Tsinghua University in 2019, and the M.E. and Ph.D. degrees in information and communications engineering from Tokyo Institute of Technology in 2021 and 2024, respectively. His research focuses on signal processing, optimization, and machine learning. His recent research explores combining diffusion models with model-based inference algorithms for inverse problems. He received the 2024 IEEE SPS Japan Student Journal Paper Award and the 2025 Seiichi Tejima Doctoral Dissertation Award.

תחילת שנת הלימודים האקדמית תשפ"ז 2026-2027

עצרת לציון 30 שנים להרצחו של יצחק רבין

Abstract: Foundation models present a paradigm shift in machine learning, where it often appears that “scale is all you need” to achieve strong performance across tasks. However, the exponential growth of these models demands substantial memory and computational resources, limiting their usability in small research environments. In this talk, I will discuss our ongoing efforts to make training or fine-tuning large foundation models more efficient and accessible. I will first introduce AdaRankGrad, which exploits the low-rank structure of gradients to enable memory-efficient full-parameter fine-tuning. I will then present SUMO, a subspace-aware optimizer that performs exact moment orthogonalization to accelerate convergence and enhance generalization while further reducing memory requirements. Finally, I will discuss a complementary approach that compresses models during fine-tuning through stochastic gating, yielding compact networks that maintain accuracy while reducing inference time. The talk is based on: [1] Yehonathan Refael, Jonathan Svirsky, Boris Shustin, Wasim Huleihel, and Ofir Lindenbaum. "AdaRankGrad: Adaptive Gradient Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning." ICLR, 2025. [2] Yehonathan Refael, Guy Smorodinsky, Tom Tirer, and Ofir Lindenbaum. "SUMO: Subspace-Aware Moment-Orthogonalization for Accelerating Memory-Efficient LLM Training." NeurIPS, 2025. [3] Jonathan Svirsky, Refael Yehonathan, and Ofir Lindenbaum. “Train Less, Infer Faster: Efficient Model Finetuning and Compression via Structured Sparsity.” AISTATS, 2026. Bio: Ofir Lindenbaum is a senior lecturer (assistant professor) in the Faculty of Engineering at Bar-Ilan University. He completed a postdoctoral fellowship at Yale University in the Applied Mathematics Program, working with Prof. Ronald Coifman and Prof. Yuval Kluger, and earned his Ph.D. in Electrical Engineering at Tel Aviv University under the supervision of Prof. Arie Yeredor and Prof. Amir Averbuch. His research focuses on developing machine learning methods to advance scientific discovery. He works on interpretable, efficient models for high-dimensional tabular data and multimodal learning, and addresses questions in sparsification, optimization, and representation learning. His work aims to develop principled, reliable, and data-efficient algorithms to extract meaningful structure from real-world scientific measurements.