I think I may switch back to Twitter/X. Somehow I feel this site didn't take off and I really don't want to be looking at two feeds all the time...

Image matching and ChatGPT - new post in the wide baseline stereo blog. tl;dr: it is good, even feels like human, but not perfect. ducha-aiki.github.io/wide-baselin...

Many of the greatest papers, now canonical works, have a story of resistance, tension, and, finally, a crucial advocate. It's shockingly common. Why is there a bias against excellence? And what happens to those papers, those people, when no one has the courage to advocate?

Do you want to work with me for some months? Two internship positions available at the Music Team of Sony AI in Barcelona! 👇

I'm happy to have two papers accepted at #ICASSP2025! 1) Contrastive learning for audio-video sequences, exploiting the fact that they are *sequences*: arxiv.org/abs/2407.05782 2) Knowledge distillation at *pre-training* time to help generative speech enhancement: arxiv.org/abs/2409.09357

Flow matching mapping text to image directly (instead of noise to image): cross-flow.github.io

With some delay, JetFormer's *prequel* paper is finally out on arXiv: a radically simple ViT-based normalizing flow (NF) model that achieves SOTA results in its class. Jet is one of the key components of JetFormer, deserving a standalone report. Let's unpack: 🧵⬇️

Did you miss any of the talks of the Deep Learning Barcelona Symposyum 2024 ? Play them now from the recorded stream: www.youtube.com/live/yPc-Un3...

I'll get straight to the point. We trained 2 new models. Like BERT, but modern. ModernBERT. Not some hypey GenAI thing, but a proper workhorse model, for retrieval, classification, etc. Real practical stuff. It's much faster, more accurate, longer context, and more useful. 🧵

On pre-acrivation norm, learnable residuals, etc.

Two great tokenizer blog posts that helped me over the years: sjmielke.com/papers/token... sjmielke.com/comparing-pe... People have mostly standardized on certain tokenizations right now, but there are huge performance gaps between locales with high agglomeration (e.g. common en-us) and ...

Don't be like Reviewer 2.

Did Gauss invent the Gaussian? - Laplace wrote down the integral first in 1783 - Gauss then described it in 1809 in the context of least-sq. for astronomical measurements - Pearson & Fisher framed it as ‘normal’ density only in 1910 * Best part is: Gauss gave Laplace credit!

I already signed up (as a mentor) for this year!

Thrilled to present our work on Constrained Parameter Regularization (CPR) at #NeurIPS2024! Our novel deep learning regularization outperforms weight decay across various tasks. neurips.cc/virtual/2024... This is joint work with Michael Hefenbrock, Gregor Köhler, and Frank Hutter 🧵👇

Entropy is one of those formulas that many of us learn, swallow whole, and even use regularly without really understanding. (E.g., where does that “log” come from? Are there other possible formulas?) Yet there's an intuitive & almost inevitable way to arrive at this expression.

Inventors of flow matching have released a comprehensive guide going over the math & code of flow matching! Also covers variants like non-Euclidean & discrete flow matching. A PyTorch library is also released with this guide! This looks like a very good read! 🔥 arxiv: arxiv.org/abs/2412.06264

Normalizing Flows are Capable Generative Models Apple introduces TarFlow, a new Transformer-based variant of Masked Autoregressive Flows. SOTA on likelihood estimation for images, quality and diversity comparable to diffusion models. arxiv.org/abs/2412.06329

That was fast: #DLBCN 2024 was sold out in less than two hours ! New requests will be added to a waiting list. Read the instructions for same day event registration: sites.google.com/view/dlbcn20...

Past work has characterized the functions learned by neural networks: arxiv.org/pdf/1910.01635, arxiv.org/abs/1902.05040, arxiv.org/abs/2109.12960, arxiv.org/abs/2105.03361. But it turns out multi-task training produces strikingly different solutions! Adding tasks produces “kernel-like” solutions.

Can language models transcend the limitations of training data? We train LMs on a formal grammar, then prompt them OUTSIDE of this grammar. We find that LMs often extrapolate logical rules and apply them OOD, too. Proof of a useful inductive bias. Check it out at NeurIPS: nips.cc/virtual/2024...

"We therefore recommend to consider logistic regression as the first choice for data-scarce applications with tabular data and provide practitioners with best practices for further method selection." arxiv.org/abs/2405.07662

🤔 Why do we extract diffusion features from noisy images? Isn’t that destroying information? Yes, it is - but we found a way to do better. 🚀 Here’s how we unlock better features, no noise, no hassle. 📝 Project Page: compvis.github.io/cleandift 💻 Code: github.com/CompVis/clea... 🧵👇

My eyes, it hurts.

New paper shows AI art models don't need art training data to make recognizable artistic work. Train on regular photos, let an artist add 10-15 examples their own art (or some other artistic inspiration), and get results similar to models trained on millions of people’s artworks

Nice to see this article by @mbaldwin.bsky.social surface again. Lots of academics aren't aware how recent and dynamic the peer-review system is. Entire thing worth a read, but this quote always floors me (page 8 of PDF ethos.lps.library.cmu.edu/article/19/g... ):

For those of you who don't know (or don't remember), the original paper introducing "attention" was Bahdanau, Cho, & Bengio's "Neural machine translation by jointly learning to align and translate" (2014), not the famous "Attention is all you need" from 2017. arxiv.org/abs/1409.0473

Better VQ-VAEs with this one weird rotation trick! I missed this when it came out, but I love papers like this: a simple change to an already powerful technique, that significantly improves results without introducing complexity or hyperparameters.

Have you ever wondered how to train an autoregressive generative transformer on text and raw pixels, without a pretrained visual tokenizer (e.g. VQ-VAE)? We have been pondering this during summer and developed a new model: JetFormer 🌊🤖 arxiv.org/abs/2411.19722 A thread 👇 1/

I learned about this paper (arxiv.org/abs/2406.09413) when Alexei gave this wonderful talk at the U. They trained 60K diffusion models, each for a different person's visual identity. Sampling weights from this set creates a model for a novel identity.

A paper a day, episode 15. You liked the matrix cookbook? You’re gonna love this one. 100 statistics inequalities just for your personal enjoyment. As they say in French, moi j’ai Bienaymé cet article ! arxiv.org/abs/2102.07234

I wrote a summary of the main ingredients of the neat proof by Hugo Lavenant that diffusion models do not generally define optimal transport. github.com/mathematical...

Excellent explanation of RoPE embedding, from scratch with all the math needed: https://fleetwood.dev/posts/you-could-have-designed-SOTA-positional-encoding And with beautiful 3blue1brown's style of animation: https://github.com/3b1b/manim. Original RoPE paper: arxiv.org/abs/2104.09864

We are now also on Instagram. We will use this channel to explain what deep learning is to our local community. Follow us at @dlbcn.ai

iNatSounds: new dataset from folks @inaturalist.bsky.social & co-authors; looks to be one of the largest public datasets of animal sounds openreview.net/forum?id=QCY... github.com/visipedia/in... #prattle 💬 #bioacoustics

Very cool/curious finding!

Amazing blog post on flow matching, stunning visuals! It also makes the connection with normalising flows crystal clear. Incredible effort!

".. these findings are insights that people in this field have intuitively understood without the need for experiments .." reject!

Scholar-inbox is quite good IMHO.

Arxiv sharing reminder pdf ❌ abs ✅

My deep learning course at the University of Geneva is available on-line. 1000+ slides, ~20h of screen-casts. Full of examples in PyTorch. fleuret.org/dlc/ And my "Little Book of Deep Learning" is available as a phone-formatted pdf (nearing 700k downloads!) fleuret.org/lbdl/

Audio bandwidth extension (why do people call this super-resolution?) is making quite some progress! demo: aeromamba-super-resolution.github.io code: github.com/aeromamba-su...

Hello everyone!