I'm in Suzhou to present our work on MultiBLiMP, Friday @ 11:45 in the Multilinguality session (A301)! Come check it out if your interested in multilingual linguistic evaluation of LLMs (there will be parse trees on the slides! There's still use for syntactic structure!) arxiv.org/abs/2504.02768

🌍Introducing BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data! LLMs learn from vastly more data than humans ever experience. BabyLM challenges this paradigm by focusing on developmentally plausible data We extend this effort to 45 new languages!

Unfortunately, our submission to #NeurIPS didn’t go through with (5,4,4,3). But because I think it’s an excellent paper, I decided to share it anyway. We show how to efficiently apply Bayesian learning in VLMs, improve calibration, and do active learning. Cool stuff! 📝 arxiv.org/abs/2412.06014

Accepted to NeurIPS! 😁 We will present Neurosymbolic Diffusion Models in San Diego 🇺🇸 and Copenhagen 🇩🇰 thanks to @euripsconf.bsky.social 🇪🇺

A really neat paper thinking through what "Identifiability" means, how we can determine it, and implications for modeling. Arxiv: arxiv.org/abs/2508.18853 #statssky #mlsky

The most expensive part of training is the data, not the compute Nikhil Kandpal & Colin Raffel calculate a really low bar for how much it would cost to produce LLM training data with 3.8$\h Well, several scales more than the compute. Luckily (?), companies don't pay for the data 🤖📈🧠

Organising NeSy 2025 together with some of my favourite people (@e-giunchiglia.bsky.social @lgilpin.bsky.social @pascalhitzler.bsky.social) really was a dream. Super proud of what we've achieved! See you next year 😘

Our second keynote speaker is @tkipf.bsky.social, who will discuss object-centric representation learning! Do objects need a special treatment for generative AI and world models? 🤔 We will hear on Monday!

It is almost time to welcome you all in Santa Cruz! 🦕 We will start with an exciting and timely keynote by @guyvdb.bsky.social on "Symbolic Reasoning in the Age of Large Language Models" 👀 📆 Full conference schedule: 2025.nesyconf.org/schedule/

Does a smaller latent space lead to worse generation in latent diffusion models? Not necessarily! We show that LDMs are extremely robust to a wide range of compression rates (10-1000x) in the context of physics emulation. We got lost in latent space. Join us 👇

We are immensely excited to announce our 4 leading researchers within our community as the #EurIPS keynote speakers🎉 @ulrikeluxburg.bsky.social Michael Jordan Emtiyaz Khan Amnon Shashua More details to come as we get closer to December, so stay tuned

Ironic: as LLMs make writing much easier, I am finding not more, but less interesting and novel things to read online. And so I keep paying more attention to the fewer people who still write their original thoughts (without LLMs - you can tell how repetitive it gets with them)

This is a good feed to have in the mixer. It shows all papers being shared by your follows

Free online Neuro-Symbolic AI Summer School today and tomorrow. Looks like some interesting talks!

We are excited to bring #EurIPS 2025 to Copenhagen in December. Consider becoming a sponsor and support us in making this inaugural event a success! Sponsorship packages are available and can be further customized if necessary. Reach out if you have any questions ❔ Info: eurips.cc/become-spons...

Thank you! Somebody (@gracekind.net ) has finally found the perfect way to answer these "its just matrix multiplication" takes. Also, the take "there is nothing new with deep learning, neural nets were around 50y ago" is like "nothing new with humans, amino acides were around 4.4 billion y ago".

GPT-5 lands first place on NoCha, our long-context book understanding benchmark. That said, this is a tiny improvement (~1%) over o1-preview, which was released almost one year ago. Have long-context models hit a wall? Accuracy of human readers is >97%... Long way to go!

We built the simplest possible social media platform. No algorithms. No ads. Just LLM agents posting and following. It still became a polarization machine. Then we tried six interventions to fix social media. The results were… not what we expected. arxiv.org/abs/2508.03385

It's been a while, but I am happy to share that my PhD dissertation is finally available online! 🎉 Not only it contains most of my work, but there is plenty of brand new content: publikationen.sulb.uni-saarland.de/handle/20.50... 🧵1/4

hey wasn't this the same company that made a beautiful shiny "research" post about how AI evals should include error bars or something like that. or did they decide the CLT didn't apply here

A reminder that the 70+ year-old research discipline of AI is much bigger than genAI, much bigger than just deep learning, much bigger than Big Tech. AI includes symbolic reasoning, multi-agent systems and statistical machine learning and is used for astrophysics, renewable energy and climate models

Our illustrated guide to non-Euclidian ML is finally published! Check it out for ⭐️ gorgeous figures (with new additions!) on topology, algebra, and geometry in the field ⭐️ broken down tables for easy reading ⭐️ accessible text, additional refs, and more iopscience.iop.org/article/10.1...

Supermassive congrats to Giwon Hong (@giwonhong.bsky.social) for the amazing feat! 🙂

and to @leanderk.bsky.social @paolomorettin.bsky.social Roberto Sebastiani, @andreapasserini.bsky.social @nolovedeeplearning.bsky.social for the ✨Best Student Paper Runner Up Award✨ for "A Probabilistic Neurosymbolic Layer for Algebraic Constraint Satisfaction" 👉 openreview.net/forum?id=9Uk...

BPE is a greedy method to find a tokeniser which maximises compression! Why don't we try to find properly optimal tokenisers instead? Well, it seems this is a pretty difficult—in fact, NP-complete—problem!🤯 New paper + @philipwitti.bsky.social @gregorbachmann.bsky.social :) arxiv.org/abs/2412.15210

🍇Happy to share GRAPES 🍇 Make GNNs work on large graphs by learning an expressive and adaptive sampler 🚀 Excellent work led by Taraneh Younesian now in TMLR! openreview.net/forum?id=QI0...

Rule learning is valuable but overlooked in modern AI; we argue it merits more focus for two reasons: knowledge discovery and interpretable inference. Our NeSy 2025 paper presents a state-of-the-art neurosymbolic tabular rule learner. 📄 arxiv.org/pdf/2504.19354 🐍 github.com/DiTEC-projec... 🧵1/8

🍇Happy to share GRAPES 🍇 Make GNNs work on large graphs by learning an expressive and adaptive sampler 🚀 Excellent work led by Taraneh Younesian now in TMLR! openreview.net/forum?id=QI0...

I love the #eurips initiative! But to live up to its potential, it should be accepted an official #neurips conference location (similar to Mexico City) and not just an addon! Then we would save C02, rather than adding to it!!! What can we do to achieve this?

Maybe an important nuance here is that the causal description doesn't necessarily have higher loss - there are just many solutions with low loss, and identifying the causal one is hard! See eg the work on Neurosymbolic Reasoning Shortcuts arxiv.org/abs/2305.19951

Beautifully succinct statement of the core problem with the blind application of statistical learning in AI!!

Hmm, this rule seems very recently changed? web.archive.org/web/20250514... This was what it said just before the deadline, so people submitting to NeurIPS couldn't have known about the mandatory physical presence 🤔

✨Thrilled to see EurIPS launch — the first officially endorsed European NeurIPS presentation venue! 👀 But NeurIPS now requires at least one author to attend in San Diego or Mexico (and not just virtually as before). This is detrimental to many. Why not allow presenting at EurIPS or online? 1/4

Exciting development for Europe! However I am still puzzled about "Note that accepted authors should still plan to present their papers at NeurIPS, with an additional presentation at EurIPS if desired". How can we be at both places at once?

EurIPS is coming! 📣 Mark your calendar for Dec. 2-7, 2025 in Copenhagen 📅 EurIPS is a community-organized conference where you can present accepted NeurIPS 2025 papers, endorsed by @neuripsconf.bsky.social and @nordicair.bsky.social and is co-developed by @ellis.eu eurips.cc

🚨 New preprint! 🚨 Everyone loves causal interp. It’s coherently defined! It makes testable predictions about mechanistic interventions! But what if we had a different objective: predicting model behavior not under mechanistic interventions, but on unseen input data?

🚀 Big time! We can finally do simple LLM RL fine-tuning with rewards and leverage offline/off-policy data! ❌ You want rewards, but GRPO only works online? ❌ You want offline, but DPO is limited to preferences? ✅ QRPO can do both! 🧵Here's how we do it:

I am in Vancouver at ICML, and tomorrow I will present our newest paper "Partially Observable Reinforcement Learning with Memory Traces". We argue that eligibility traces are more effective than sliding windows as a memory mechanism for RL in POMDPs. 🧵

At #ICML, you can just use scholar inbox to help you find your way through the poster sessions. It just sorts the papers according to your preferences and it really works. www.scholar-inbox.com/conference/i... ICML 2025 Planner

🚨 First Call for Participation – NeSy 2025 📍 Sept 8–10 | Santa Cruz, CA Join the longest-running conference on neurosymbolic AI! Our keynote speakers: @guyvdb.bsky.social @tkipf.bsky.social Deborah McGuinness @garymarcus.bsky.social More info 👇

Good news! Trust in science is going up slightly in the Netherlands. Also in recent years 👏 Now let's invest in science too!! New report from Rathenau with numbers from 2012-2025. www.rathenau.nl/nl/werking-v...

Accelerated Diffusion Models via Speculative Sampling, at #icml25 ! 16:30 Tuesday July 15 poster E-3012 arxiv.org/abs/2501.05370 @vdebortoli.bsky.social Galashov @arnauddoucet.bsky.social

Spotlight poster coming soon at #ICML2025 @icmlconf.bsky.social! 📌East Exhibition Hall A-B E-1806 🗓️Wed 16 Jul 4:30 p.m. PDT — 7 p.m. PDT 📜 arxiv.org/pdf/2410.12537 Let’s chat! I’m always up for conversations about knowledge graphs, reasoning, neuro-symbolic AI, and benchmarking.

We ran a randomized controlled trial to see how much AI coding tools speed up experienced open-source developers. The results surprised us: Developers thought they were 20% faster with AI tools, but they were actually 19% slower when they had access to AI than when they didn't.

First position paper I ever wrote. "Beyond Statistical Learning: Exact Learning Is Essential for General Intelligence" arxiv.org/abs/2506.23908 Background: I'd like LLMs to help me do math, but statistical learning seems inadequate to make this happen. What do you all think?

👨‍🎓🧾✨#icml2025 Paper: TabICL, A Tabular Foundation Model for In-Context Learning on Large Data With Jingang Qu, @dholzmueller.bsky.social, and Marine Le Morvan TL;DR: a well-designed architecture and pretraining gives best tabular learner, and more scalable On top, it's 100% open source 1/9

Our new ICML paper discovers scaling collapse: through a simple affine transformation, whole training loss curves across model sizes with optimally scaled hypers collapse to a single universal curve! We explain the collapse, providing a diagnostic for model scaling. arxiv.org/abs/2507.02119 1/3

I really liked that short blog post by Sergey Levine on the fundamental difference between learning LLMs and video prediction. sergeylevine.substack.com/p/language-m...

"in 2025 we will have flying cars" 😂😂😂