I asked "on the other platform" what were the most important improvements to the original 2017 transformer. That was quite popular and here is a synthesis of the responses:

"You are in Paris, enjoy the city, stop obsessing with AI" Paris:

Maybe the wall was the friends we made during that journey Ted.

What is the true depth of an LLM? Together with @danielepal.bsky.social , @matpagliardini.bsky.social, M. Jaggi and @francois.fleuret.org we show that LLMs have a smaller effective depth that can be exploited to increase inference speeds on multi-GPU settings! arxiv.org/abs/2502.02790 (1/N)

J'étais l'invité du journal de 19h30 sur la @radiotelesuisse.bsky.social ce soir pour parler d'Intelligence Artificielle. www.rts.ch/play/tv/19h3...

It is hard to overstate how cool and powerful is flex attention. @chhillee.bsky.social pytorch.org/blog/flexatten… TL;DR: it is an implementation of the attention operator in pytorch that allows in particular to efficiently "carve" the attention matrix. 1/3

Finally got this beautiful piece from @francois.fleuret.org

Happy new year you all! 2025 is certainly full of promise.

Happy Christmas you all!

Whatever you say about the whole field of AI: It's not boring.

Some tools that keep me sane on Mac: rectangleapp.com karabiner-elements.pqrs.org

It's Friday!

Oh boy, GTA 6 has to be good. And Half Life 3.

This is very great.

You young post-2012 researchers, you realise that us the old people are *amazed* by what we witness?

Something that annoys me much is when people (generally in political / "societal" discussions) equate the validity of a statement with the validity of *what "people" may understand* and generally then fix a true statement they consider false with a false statement they consider true. 1/2

hello bluesky! we have a new preprint on solvation free energies: tl;dr: We define an interpolating density by its sampling process, and learn the corresponding equilibrium potential with score matching. arxiv.org/abs/2410.15815 with @francois.fleuret.org and @tbereau.bsky.social (1/n)

"The expletive error rate is the number of times you call your algorithm a 'dense mf' when looking at one run"

General structure of a paper: - general ideas - general case - general case - general case - what we actually do how it should be: - what we actually do - why we think it's great as one method of a general class - how we got there - how we got there - how we got there

Today's mood "wow GPTs are dumb"

One of the goosebumpsiest short movie ever (where this quote comes from). We will have this sound snippet played at the ceremony when we will leave earth for the stars. www.youtube.com/watch?v=YH3c...

"We invest far-off places with a certain romance. This appeal, I suspect, has been meticulously crafted by natural selection as an essential element in our survival."

If you have not seen it, you really should. m.youtube.com/watch?v=WXuK...

🎶 This is the end 🎶 @neuripsconf.bsky.social

Just gave a talk on "Grounding LLMs in Code Execution" at the NeurIPS Hacker-Cup AI Competition, here are the slides docs.google.com/presentation...

That The shining vibe.

My political weight is obviously immense since there is coffee available all the time at @neuripsconf.bsky.social this year. For next year: can we have at e.g. 10 places in the center some huge colorful discs with numbers 1...10 in them to serve as convenient meeting points?

8th gen

The topologists seem to be taking over. Large pictures + a README about this experiment at fleuret.org/public/culture

Try 💎 DIAMOND’s Counter Strike world model directly in your browser! → next.journee.ai/xyz-diamond ← How long can you stay in distribution? Can you beat @snguyen.bsky.social’s 1000 frames? @eloialonso.bsky.social and I are at NeurIPS! Poster #6306, Friday 11am-2pm, West Ballroom

That Kandinsky vibe.

I like north America but it's not *normal* and I suspect not *good* to always sleep with an air-conditioning that sounds like a jumbo jet taking off.

Can you simplify that one? github.com/francoisfleu...

Chicken or pasta?

Those people at Zurich airport look like NeurIPS attendees. All of them.

This is totally reasonable.

That exhilarating feeling that *everything is possible* when you open an editor to code, it hopefully never goes away.

It's Monday! On my way to @neuripsconf.bsky.social !

Can we have a TL;DR of what happened in Syria? Why did the regime fall now?

Given m a NxT boolean tensor and x a NxTxD tensor, how to write the following efficiently in PyTorch: If for a certain n the non-zeroes are t_1 ... t_K, I want x'[n,0] = x[n,t_1] x'[n,t_1] = x[n, t_2] ... x'[n, t_K-1] = x[n, t_K] x'[n, t_K] = x[n, T-1] the others x' are zeroed ?

So I wrote this sparsifying layer: I get a NxTxD tensor, compute the L1 norms of the NxT vectors, and set to zero the 95% with lesser norms. A tiny convnet with this stuff in the middle learns alright and I find it quite puzzling.

The legend foretold that one day there will be a nerd, the nerd, that will be perfectly clear about what goes in the .bash_profile and what goes in the .bashrc.

So it happens that having a file "queue.py" in the current dir when you import pytorch brings you into bizarro world. Seriously?

And it goes both ways, it is exhausting. I have a great idea to fix a problem, write a minimal toy experiment that would exhibit the said problem, and there is no problem. I engineer that minimal setup to have that problem appear. Nope, still no problem. The gradient is as happy as ever.

My brain after years of research in ML/DL is a wasteland of broken intuition pumps. Seriously NOTHING BEHAVES LIKE IT SHOULD.