🤹 New blog post! I write about our recent work on using hierarchical trees to enable sparse attention over irregular data (point clouds, meshes) - Erwin Transformer, accepted to ICML 2025 blog: maxxxzdn.github.io/blog/erwin/ paper: arxiv.org/abs/2502.17019 Compressed version in the thread below:

Can only speak for my ICML reviewing batch, but the hack of putting scary, convoluted and wrong math still works.

🤹 Excited to share Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systems joint work with @wellingmax.bsky.social and @jwvdm.bsky.social preprint: arxiv.org/abs/2502.17019 code: github.com/maxxxzdn/erwin

Interestingly, something similar can be done for irregular data structures - point clouds - with a couple of tricks including hierarchical trees and FlexAttention🌳 1/N 🧵

the number of snarky comments about Musk from anime pfps on Twitter lately left me wondering how much of the right turn of the group is just teenage contrarianism

🚨 Excited to share our work just published in Physical Review Letters with @wellingmax.bsky.social, @jwvdm.bsky.social, @berndensing.bsky.social, Marjolein Dijkstra and René van Roij: doi.org/10.1103/Phys.... Details below 👇

jax is very aesthetic until you need to do masked select

This is how it should be: one file, ctrl+c ctrl+v runnable from a notebook. I really do not want to mess with omegaconf, hydra or, the worst, a custom configuration system just to figure out how your model runs.

The comments section is depressing to read. I guess automation is inevitable in a progressing society, but such incidents do bring a bitter flavour to the current advancements.

grabbing the whole node on a cluster for an interactive session for a week should be considered antisocial behavior

🚨 FINAL REMINDER 🚨: Multiple Postdoc and PhD positions in our AI + Physics cluster in DAMTP, Cambridge! Deadlines: - Postdoc: Jan 5th (Sunday) - PhD: Jan 7th (Tuesday) More info below – Please share with researchers and students who might be interested in joining us!

tired: graphs wired: trees

Best lab account around here

🚨 PhD position alert! 🚨 I'm hiring a fully funded PhD student to work on mechanistic interpretability at @uva-amsterdam.bsky.social. If you're interested in reverse engineering modern deep learning architectures, please apply: vacatures.uva.nl/UvA/job/PhD-...

If you are attending #NeurIPS2024🇨🇦, make sure to check out AMLab's 11 accepted papers ...and to have a chat with our members there! 👩‍🔬🍻☕ Submissions include generative modelling, AI4Science, geometric deep learning, reinforcement learning and early exiting. See the thread for the full list! 🧵1 / 12

I will be at NeurIPS this week. @amlab.bsky.social will be presenting 10 papers in total, and I myself am involved with 3 (see thread). I am also *hiring* for a postdoc position on data-efficient surrogate models for fluid dynamics. Come talk to me if you are on the market!

At a certain point, I was so inspired by the geomancer paper by @davidpfau.com et al. that I decided to apply for a PhD in causality/GDL at ELLIS. All those buzzwords in my motivational letter just to find later that I was writing about casual ML all along 🤦‍♂️

hi everyone!! let's try this optimal transport again 🙃

Yesterday @ellisamsterdam.bsky.social hosted the yearly NeurIPS-Fest, a pre-party for NeurIPS with a keynote talk, poster session, drinks and bites! 🍺🍻 The keynote was by @canaesseth.bsky.social , who talked about "Diffusion, Flows and other stories", presenting his 5 papers accepted at NeurIPS! 💥

Fun paper led by Julia Balla: "A Cosmic-Scale Benchmark for Symmetry-Preserving Data Processing". Julia will be presenting the paper at LoG on Thursday as a spotlight oral, and also at the NeurReps Workshop at NeurIPS Workshop next month. 📄: arxiv.org/abs/2410.20516 💻: github.com/smsharma/eqn...

Arvind, me, and Jonah released a new pre-print on some pen and paper analysis of fundamental failure modes and old school stability analysis for neural PDEs typically used in AI for Science application. arxiv.org/abs/2411.15101. 1/n

Looking at ICLR submissions with the lowest score - What a work of art! 🧵

Meet our Lab's members: staff, postdocs and PhD students! :) With this starter pack you can easily connect with us and keep up to date with all the member's research and news 🦋 go.bsky.app/8EGigUy

New year's resolution: make it to the grumpy ML starter pack

Hi everyone! This is AMLab :) Looking forward to share our research here on 🦋 !

Yesterday was my first day at FAIR Chemistry as a research scientist. Looking for interns for summer 2025!

I was playing with the fast multipole method this summer, mainly by translating the fantastic pybbfmm library from torch to jax. showcase: simulation of an infection spread over the British Isles (10M agents, 3s to simulate with jax, 3x over pytorch).