A nice paper in ICML2025, generalizing the smoothness notion, for which GD convergence is provided. The convergence proof is very transparent: descent lemma + telescoping + estimation of stepsize. arxiv.org/abs/2412.11773

Yesterday I learned from a pharmacologist that an integration of surrogate models and pharmacokinetics (PK) has been emerging. For example this (jpharmsci.org/article/S002... ). I just wonder if we can refine it by PINN-like approach.

Thrilled to share our paper is accepted to #NeurIPS2025 as a Spotlight! 🎉 Big thanks to my awesome collaborators and the program committee. See you in San Diego!

Why don't we implement ACL Findings type publication models to the ML venues? I don't see any outstanding reason to be reluctant. I know borderline papers sometimes do not provide "surprising" insights, but this ought to be judged by the test of time.

This is the metareview

This is not OK. I don't submit often to NeurIPS, but I reviewed papers for this conference almost every year. As a reviewer, why would I spend time trying to give a fair opinion on papers if it's what happens in the end???

Je suis tellement content que mon papier est accepté à JMLR après plus d’un an! C’était très long…

In this weekend I attended a workshop among mathematicians, where I was discussing dynamical systems. During the workshop, there are numerous novel insights to me, which I want to briefly share. 1) metastability in Allen-Cahn equation (fig based on people.maths.ox.ac.uk/trefethen/pd...)

Today's my favorites: "Clustering with Bregman Divergences: an Asymptotic Analysis" (Liu & Belkin, 2016) proceedings.neurips.cc/paper_files/... It's concerned with the limiting distribution of k-means (or Bregman) centroids (w/ n, k -> \infty). This is an escort distribution! (maybe overlooked)

Today I learned that the continuous-time limit of Nesterov's accelerated gradient is Bessel's differential equation, which can be solved analytically. That's unexpectedly a beautiful result to me... web.stanford.edu/~boyd/papers...

I was on a three-day vacation in Siem Reap 🇰🇭 Here are a few views of my favorites: 1) Baphuon

I participated in Deep Learning Theory Workshop 2025 held in RIKEN AIP. I enjoyed lively discussions among the participants. delta-workshop.github.io/deep-learnin...

I thought IIIT Hyderabad and IIIT Delhi are branches of IIIT (not IIT to make sure!), but it turns out to be that the first I are different!

I'm recently learning MPEC (mathematical programming with equilibrium constraints), which is basically a bilevel programming. Since MPEC is known to violate LICQ (I didn't realize it before), it's much harder. Interestingly, MPEC-ver of LICQ etc. is known instead. link.springer.com/article/10.1...

AISTATS 2026 will be in Morocco!

I’m so glad to get passed French DELF A1! (Still need to train my listening skill quite a lot 🫠)

NeurIPS satellite and EurIPS are interesting experiments and potentially mitigate visa issues and environmental concerns for sure. Nevertheless, Asian local communities are oftentimes not associated well with the "global" academic community. I feel our geographic location remains to be an obstacle.

The unsustainability of long distance flights to conferences is a valid concern. Clearly, meeting international experts is important for high-level research, but dynamic regional events to supplement huge monolithic conferences might even make that easier. We need more experiments like EurIPS asap.

Je suis arrivé à Lyon le jour de 125e anniversaire de Saint-Ex!

Sunset view at ISM this evening

Recently I get to recognize there are at least three kinda independent streams in research on large stepsize GD: (1) edge of stability (e.g. openreview.net/pdf?id=jh-rT...), (2) silver stepsize (e.g. arxiv.org/abs/2309.07879), (3) minima stability (e.g. proceedings.neurips.cc/paper/2021/f...)

J'ai fini mon premier examen de français (après un an d'études). Pas sûr de le passer mais je l'espère.

Finally got my office at ISM! (the name correspondence does not look great tho 😂 Han is my first name corresponding to 含 in Kanji)

Today I met a political scientist who is familiar with Wasserstein distance, which was quite amazing...

On this Friday, Yutong@Illinois Tech will visit ISM and give a talk on the recent advances in multiclass classification theory, including implicit bias and the connections to SVM, ranking, noisy labels. Please check it out (you can view it online). www.ism.ac.jp/events/2025/...

Folks, here's our latest work on convolutional Fenchel-Young losses! We show convex smooth losses can have a linear surrogate regret bound on discrete losses (e.g. 01 loss, Prec@k), and it's relies on inf-conv, which is the most beautiful proof I saw ever. arxiv.org/abs/2505.09432

I’m highly delighted that finally I can attend AISTATS in person (I had AISTATS papers during online years only so far😂). The community size remains to be quite reasonable and people are enthusiastic even after the poster session! (Someone stopped me for asking about our poster)

pretty unexpected rejection... c'est la vie😵‍💫

I’ve finished almost all of research talks in Singapore during the long long one week! Here’s the Hive, a very cool architecture by the Heatherwick Studio. Finally I encountered this after finishing the talk at NTU 😊

#ICLR2025 We are going to present it this evening. If you are interested in any of contrastive representation learning, predictive coding, learning dynamics, come by at #608!

I feel like briefly mentioning my favorite paper a day in ICLR. [Day 1] The two-layer ReLU nonconvex objective can be elegantly reformulated into a convex objective. Therein we can invoke convex analysis to characterize the phase transition of mode connectivity. openreview.net/forum?id=4xW...

I'm glad to be chosen as a recipient of AISTATS2025 Best Reviewer Award🥳🥳🥳 Among ML confs, I quite love the atmosphere of AISTATS. (Although this year I'm going to attend AISTATS in person for the first time🤣) aistats.org/aistats2025/...

A new entmax-based SSL, E2MC (will appear in AISTATS25): arxiv.org/abs/2411.15931 They argued in intro > By maximizing the amount of information retained, one hopes to prepare as well as possible for future which I agree partially yet then why we need compression at all? (my everytime question🤯)

Je vais à singapour la semaine prochaine pour assister à ICLR 🇸🇬 1) TAID openreview.net/forum?id=cqs... : distillation pour modèle de langage 2) PhiNets openreview.net/forum?id=5tj... : representation learning basé sur modèle d'hippocampe

C’est très difficile de finir de travail de l’area chair… c’est la première fois pour moi

A suggested reading by my colleague on Bourgain's conjecture and Kannan-Lovasz-Simonovits conjecture, a longtime open problem in high-dim convex geometry www.quantamagazine.org/statistics-p...

No positional encoding is more common now? 🤯 (memo for reading) arxiv.org/abs/2305.19466

While reading Ben Recht's article, I found Foster & Hart (2021) (arxiv.org/abs/2210.07169) quite interesting. The contribution is a proposal of always-calibrated forecaster based on a continuously-relaxed calibration measure. But I actually love their §1.1 motivating calibration.

Now’s the time to begin Villani’s textbook little by little. Let’s see how it goes after a while.

"But if risk aversion is the intended goal, why not be explicit about it and precisely declare the purported value of a probabilistic forecast?" This is a question I'm repeatedly asking as a researcher of scoring rules🤔 www.argmin.net/p/gambling-o...

View of Hikone Castle from the campus of Shiga University

Récement j’essaie de parler le français quand je rencontre des français mais je fais beaucoup d’erreurs à chaque fois😮‍💨

Today’s OIST

Finally I’m back to OIST again to attend OIST machine learning workshop!

By the way this week I moved to a new position at the Institute of Statistical Mathematics, and today I came to Okinawa to present the recent work on self-supervised learning. Feel excited🤩 —the photo is the workshop venue Okinawa Museum

An excellent work comparing MAE- and JEPA-type SSL dynamics (the latter is commonly called non-contrastive). They get a closed-form ODE of i-th feature importance. "How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networks" (NeurIPS24) arxiv.org/abs/2407.03475

A great article by Ben Recht. It always helps us to ask “You’ve assuredly falsified something when your predictions are wrong, but what exactly?” (quoted) www.argmin.net/p/nomologica...

Today I met two guys each from France and Burkina Faso, tried my poor French and got “pas mal” “presque parfait”, which is not satisfactory enough but already a good achievement for me now😂

I've finished reading Popper's "The Poverty of Historicism" for the first time, which has bunch of insights for natural scientists. In particular I love his aphorism like "observations never precede theory, but rather they are endowed with the meaning by theory," criticizing the limit of induction.

🧗‍♂️Why GD converges beyond [step size]<2/[smoothness]? We investigate loss functions and identify their *separation margin* is an important factor. Surprisingly Renyi 2-entropy yields super fast rate T=Ω(ε^{-1/3})! arxiv.org/abs/2502.04889