new year boi

NEPA: Next-Embedding Predictive Autoregression sihanxu.me/nepa/ Key ideas: - One self-supervised signal: cosine-style next-embedding prediction - Autoregression runs directly on the model's native embeddings - No pixel decoder (& loss), no contrastive pairs, no task-specific heads, no random masks

In my new blog, “Test-Time Training Done Better: From Plastic Adaptation to Elastic Memory Consolidation,” I introduce a long-context modeling architecture that learns to adapt and memorize at test time by updating a subset of the model’s weights during inference. mars-tin.github.io/blogs/posts/...

Gosh, I’m getting way too emotional writing my thesis acknowledgements...

Will be at #NeurIPS2025 (San Diego) Dec 1-9, then in the Bay Area until the 14th. Hmu if you wanna grab coffee and talk about totally random stuff. Thread with a few things I’m excited about. P.S. 4 NeurIPS papers all started pre-May 2024 and took ~1 year of polishing...so proud of the team!

Trying to decide what to do on the first day of #NeurIPS2025? Check out my, @marstin.bsky.social and @xiangyue96.bsky.social's tutorial, "The Science of Benchmarking: What's Measured, What's Missing, What's Next" on December 2 from 1:30 to 4:00pm. benchmarking.science What will we cover? 1/3

@fredashi.bsky.social and I wrote a blog for our new mechinterp paper (arxiv.org/abs/2510.13796), including many unpublished and even negative results that we found meaningful to share. An Open-Notebook Exploration of Emergent Grounding in LMs mars-tin.github.io/blogs/posts/...

Regrettably can’t attend #COLM2025 due to deadlines, but Jane and Joyce will be presenting our work. :) Jane is an exceptional undergraduate researcher and a great collaborator! Go meet her at COLM if you’re curious about her work on mechanistic interpretability, multimodality, & pragmatics!

🚀 ACL ARR is looking for a Co-CTO to join me lead our amazing tech team and drive the future of our workflow. If you’re interested or know someone who might be, let’s connect! RTs & recommendations appreciated.

Unfortunately, I’ll be missing #ACL2025NLP this year — but here are a few things I’m excited about! 👇

📣 Excited to announce SpaVLE: #NeurIPS2025 Workshop on Space in Vision, Language, and Embodied AI! Join us in San Diego to push the frontiers of spatial understanding and reasoning across CV, NLP, and robotics! 👉 space-in-vision-language-embodied-ai.github.io

#CoreCognition #LLM #multimodal #GrowAI We spent 3 years to curate 1503 classic experiments spanning 12 core concepts in human cognitive development and evaluated on 230 MLLMs with 11 different prompts for 5 times to get over 3.8 millions inference data points. A thread (1/n) - #ICML2025 ✅

New Paper Alert ‼️ Current VLMs completely fail human gaze understanding 🙀 and scaling does NO help ‼️ However, humans, since an extremely age 🧒, are extremely sensitive to other people's gaze 🙄 👀 No mentors, no labs, only pre-doc students, 111 VLMs, and we did it 😎

& @tianminshu.bsky.social (+ @marstin.bsky.social, @zhitinghu.bsky.social, ‪@lianhui.bsky.social & more) will present “SimWorld: A World Simulator for Scaling Photorealistic Multi-Agent Interactions,” an @unrealengine.bsky.social-based sim that generates unlimited/diverse urban environments: (13/14)

At Albuquerque Now :)

See you at #NAACL2025! I will talk about grounded lexicon acquisition and scaling mechanistically grounded vision language models. Happy to chat if you are around :)

Vision-Language Models are not yet pragmatically optimal. We identify 3 key failures of pragmatic competence in referring expression generation with VLMs: (1) cannot uniquely refer to the referent, (2) include excessive or irrelevant information, and (3) misalign with human pragmatic preferences.

I won’t be attending #ICLR2025 in person since #NAACL2025 follows right after, but here are a few things I’m excited about (all time in EDT) ⬇️

#ICLR2025 and #CHI2025 are just around the corner! We warmly invite you to join us at our ICLR Workshop and CHI SIG on Bidirectional Human-AI Alignment (Bi-Align), a space for rigorous and reflective conversations of alignment research.

📢 Join us for the ACL Mentorship Session @naaclmeeting.bsky.social #NAACL2025 Mentors: • @amuuueller.bsky.social • @fredashi.bsky.social • Jiayuan Mao • @marstin.bsky.social • Oana Ignat • Weijia Shi • @zhijingjin.bsky.social

Meet VEGGIE 🥦 VEGGIE is an instructional video generative model trained solely with diffusion loss, designed for both video concept grounding and instruction-based editing. It effectively handles diverse video editing tasks by pixel-level grounded training in a multi-task learning setup. ⬇️

I am looking for a postdoc. A serious-looking call coming soon, but this is to get it going. Topics include (but not limited to): LLMs (🫢!), multimodal LLMs, interaction+learning, RL, intersection with cogsci, ... see our work to get an idea: yoavartzi.com/pubs Plz RT 🙏

Our #ICLR2025 workshop on Bidirectional Human-AI Alignment now has a sister SIG at #CHI2025! 📅 Submission deadline extended to Feb 15 to welcome both ML & HCI communities. 📍 Authors may opt to present at either ICLR or CHI 2025! More info here: bialign-workshop.github.io

[RT Appreciated] I'm excited to join #ACL @aclmentorship.bsky.social as a mentor! Your ideas can help make mentorship more impactful. Let’s plan together!🚀

Submission #1 is in the bag, but we know your masterpiece is next, don’t keep us waiting. 😎 Also, consider joining the program committee and help shape the future of alignment research. Your reviews might just steer the field! forms.gle/MapXyCZhbcFr...

Happy 2025 :) mars-tin.github.io/blogs/ground...

RepL4NLP (collocated with NAACL 2025) is inviting submissions!

Excited to co-organize the #ICLR2025 Workshop on Bidirectional Human-AI Alignment (Bi-Align). Big thanks to an amazing team and speaker lineup! We call for non-archival papers: 2-page (tiny), 4-page (short), or 9-page (long). 👉 openreview.net/group?id=ICLR.… See you in Singapore :)

I first met Jaemin at the Microsoft Accelerate Foundation Models Research (AFMR) dinner during NeurIPS 2023, though I was already familiar with his papers long before that. He’s definitely a strong candidate to consider!

I'll be in Vancouver for NeurIPS from Dec 10 to 14. Happy to chat with people! Find me at - Dec 10: Future NLP Workshop with UBC NLP - Dec 12: Vector Luncheon - Dec 14: Pluralistic Alignment Workshop, presenting the VLM frame of reference analysis work with the amazing @marstin.bsky.social

A few updates :) 1. Presenting perspective-taking in machine theory of mind at the UMich tomorrow. 2. Visiting the UW on Dec 5 to talk about language grounding, let’s connect if you’re in Seattle. 3. Attending #NeurIPS2024 in Vancouver with a conference and workshop paper, DMs open!

As the semester draws to a close, we’re grateful for our guest speakers from academia and industry: Prof. Yu Su (OSU) and Dr. Shizhe Diao (Nvidia) for our CSE 595 NLP class and the Michigan AI seminar! Quick wrap-up and links to slides below:

Once again, I'm struck by how unethical and authoritarian some ARR reviews can be. How can a reviewer dismissively state, "I'm sorry, but [some new research] is not on par with [big names from the 19xx era], etc."?

People keep telling me to “focus” on just one specific area—like grounding, language acquisition, or theory of mind. It’s as if they think I’m a laser pointer, not a curious human with a PhD in having fun.

Well, this cyber migration from X is in full swing, and honestly, it feels like moving from a chaotic street market to a cozy coffee shop. ☕️