A new paper out with TaeHo Yoon and Ernest Ryu: We looked at the design of optimal fixed-point algorithms. That is, seeking to approximately solve T(y)=y using as few evaluations of the operator T() as possible. Maximally efficient methods are "minimax optimal" 1/

Lately, I have been obsessed with developing theoretically based optimization algorithms that actually attain the best practical performance. Alas, the classic model of minimax optimal methods is overly conservative; it overfits to tune its worst-case. We found a path forward 1/

Enjoyed being part of the Brin Mathematical Research Center's summer school on Scientific Machine Learning last week. Many very good talks and always nice to visit UMD!

📢 Excited to share a new paper with PhD student Thabo Samakhoana. Nonsmooth optimization often uses smoothings, nearby smooth functions or sets. Often chosen in an ad hoc fashion. We do away with ad hoc, characterizing optimal smoothings for convex cones and sublinear functions

Oh, that’s so satisfying! I stopped at the 4-norm ball thinking I had the solution as it fits the hole like a pot lid (has a perfect circle as an intersection).

Yesterday I posted a maths puzzle that AIs all failed at (thanks for running the premium versions @xy-han.bsky.social and Ernest Ryu). The puzzle just needs elementary reasoning about p-norm balls (third row on my shelf below). This thread gives the puzzle, solution, and a 3D printed demo :)

I've invented a simple, lovely math puzzle I expect every AI fails: Suppose you're a mathematical sailor at sea on a boat that has a perfectly cylindrical hole in the floor. All you brought is a collection of every p norm ball except p=2 (drat!). What do you do to cork the hole and save yourself?

very cool talk: youtu.be/K_dhTP2I2uo?... "Near-Linear Runtime for a Classical Matrix Preconditioning Algorithm" - Jason Altschuler

Lots of great questions and engagement from Wisconsin folk! They were quick at turning around and getting it online. See below: www.youtube.com/watch?v=QNfq...

Just landed in Madison! Tomorrow, I'll be sharing my work optimizing optimization methods, to and beyond minimax optimality in their SILO seminar. Will share a link to the talk on YouTube after

The optimization technique of gradient descent is like feeling your way down a mountain in the dark. You may not be able to see the way, but you’ll eventually reach the lowest point in the area. (From the archive) www.quantamagazine.org/risky-giant-...

My PhD students are awesome. They gave my fiancee(wife) and I this gorgeous cherry blossom card for our wedding and soon honeymoon in Japan <3

As an early wedding present (happening this Saturday!), my dad made me a custom shelf to hold my collection of unit norm balls! Rockafellar+Wets's thick textbook is included for reference.

New (first) paper with my student Aaron Zoll :) We consider first-order methods for a ridiculously general model: minimizing a convex composition of functions g_j(x) that vary heterogeneously in whether they are smooth, nonsmooth, convex, strongly convex or anything in between.

🎉Congrats to the 126 early-career scientists who have been awarded a Sloan Research Fellowship this year! These exceptional scholars are drawn from 51 institutions across the US and Canada, and represent the next generation of groundbreaking researchers. sloan.org/fellowships/...

PhD students set up arts and crafts to make Valentine's mailboxes and collect cards. They (slide) rule :)

Newest office addition might be the biggest computer in my department! (Assuming compute is measured by length)

Postdoctoral position at @cornelluniversity.bsky.social, "from all areas of research that advance the state of the art in data science and the health sciences, extending the reach of data-driven research into novel medical application domains" Happy to chat! academicjobsonline.org/ajo/jobs/29529

NeurIPS has released recordings of talks! I had the privilege to speak in the OPT-ML Workshop: neurips.cc/virtual/2024... My talk presents a solution to the ''minimization game'' for smooth convex problems (ie a subgame perfect method that optimally adapts to any gradients seen)

Proud Advisor Moment: One of my first advisees, Danlin Li, now a PhD at Georgia Tech, just had our paper extending her Masters thesis accepted to Math Programming! She gives a novel primal-dual way to understand the classic (primal) subgradient method Check it out: arxiv.org/abs/2305.17323

Continuing to use January's freedom, some exposition on OWL Norms: Their unit balls are all Catalan solids (every face is the same). So the dual balls are all Archimedean solids (every corner is the same) www.ams.jhu.edu/~grimmer/OWL... Files to make your own: www.printables.com/model/113805...

January has given me some great free time :) First link gives some exposition on how this binary slide rule works. Second link gives the design files if you want one. Many libraries have 3D printers to make this for you nowadays www.ams.jhu.edu/~grimmer/Sli... www.printables.com/model/113795...

The way computers store real numbers (floating point) is exactly the same as how our grandparents did math mechanically, slide rules and log scales. I'm mass producing binary slide rules to give students on day one of my "Intro to Computational Math" this Spring :)

Yue Wu (a great PhD student) posted his first paper with me to arxiv, on stochastic nonconvex nonsmooth optimization. We give unified analysis for variance-reduced prox-linear methods, identifying a neat Pareto frontier of state-of-the-art methods. Check it out: arxiv.org/abs/2412.15008

📢 #NeurIPS2024 Folk, if you like minimax optimal gradient methods, set your alarms for tomorrow morning: 9am in the West Ballroom A I'll be giving a plenary at the OPT-ML workshop on optimizing GD steps and subgame perfect methods, all developed with Alex L Wang and Kevin Shu

My collaborator Alex L Wang wrote a nice blog post about the "Performance Estimation" view of Nesterov's Accelerated Gradient Method. I think it's a nice read (glad Bluesky doesn't suppress posts for having a link): web.ics.purdue.edu/~wang5984/bl...

New work out with Alex L Wang and Kevin Shu going beyond minimax optimal gradient method design! Kim and Fessler designed an optimal method (OGM), with the best worst-case over all smooth convex problems. Alas, on easier problems, it may be slow, its worst case occurs on x^2!

Leveling up from 3D printing: I present a p=4/3 norm ball ceramic (Moreau, featured in background, likes it)

Congratulations to all @johnshopkins.bsky.social researchers participating in #NeurIPS2024! Check out all @johnshopkins.bsky.social accepted papers, tutorials, and workshops at ai.jhu.edu/news/johns-h....

I'd like to add some more spectrahedron's to my office and to serve as examples in some upcoming talks. My current working examples come from approximating the MAX-CUT polyhedron for 3x3 and 4x4 matrices. Anyone have recommendations? Details on MAX-CUT bodies below: www.ams.jhu.edu/~grimmer/Max...

A very neat result!! Opens a new frontier to push Gradient Descent's theory! Wild that we might have different optimal exponents for the obj gap convergence rates 1/T^p of Short Stepsize GD: p=1 Anytime GD: p>=1.03 Final Iterate GD: p=1.27 (conjectured) General Grad Methods: p=2

arxiv.org/abs/2411.17668 Our postdoc zihan slays another COLT open problem! proceedings.mlr.press/v247/kornows...

I made a #starterpack for computational math 💻🧮 so please 1. share 2. let me know if you want to be on the list! (I have many new followers which I do not know well yet, so I'm sorry if you follow me and are not on here, but want to - drop me a note and I'll add you!) go.bsky.app/DXdZkzV

The DeepMath2024 Conference to going on today and tomorrow at UPenn! Talks are being streamed live online at www.youtube.com/watch?v=A0Oh... for anyone around the world interested!

Each Fall I get to teach a Freshman Experience Course where a few freshman and I explore some fun math The last weeks each year, they take the lead and design and 3D print something. This year, they made loaded dice. Next week theyll be rolling and doing statistical validation

Been thinking about smoothing of nonsmooth functions lately. Printed the 1-norm function and its Moreau Envelope, a classic smoothing tied to the proximal operator (Happy to share the source. If you teach anything related to prox operators, it could be a good aid to pass around)

Some (grand) unified theory for gradient descent's stepsize design and its minimax optimal stepsizes: arxiv.org/abs/2410.16249 with Alex L Wang and Kevin Shu. It's a wild ride :)

Recent methods for nonconvex, nonsmooth Lipschitz minimization attain "Goldstein" stationarity: an average of nearby gradients is small. In a short(!) paper, we extend this to Lipschitz constraints: averaging nearby gradients satisfies KKT! optimization-online.org/2023/10/gold...

On my way to give a seminar tomorrow at @TippieAnalytics. Will be talking about new projection-free "Radial" methods. Example constraint set props are packed and ready to go :) (Plenty of polyhedrons, spectrahedrons, norm balls, and sparsity inducing regularizers)

A while back, I 3D printed some convex cones ( www.ams.jhu.edu/~grimmer/Con... ) I'm looking for more fun ones to add to my collection. If you have recommendations, please let me know :) My current "todo" list: completely positive, exponential, and sum-of-squares cone (and their duals)

Excited to be an organizer for the 6th year the conference on mathematical theory of deep neural network (DeepMath). This year at John Hopkins University, Nov. 16 - 17. Registration is finally open, save your spot: www.eventbrite.com/e/deepmath-2...

How to (optimally) design a fair/robust syllabus for optimization classes: Below is the polyhedron of all rubrics I find reasonable, which I maximize over for each student. Lets evaluation focus on individual strengths (and nicely motivates linear programs) Details: www.ams.jhu.edu/~grimmer/Gra...