It's #NSF #GRFP application season again so it's time to re-up my GRFP application advice post! Also, check out the cool bsky comment integration I've added to the blog! Engagement with this post will go under the blogpost on my site as comments! saxon.me/blog/2024/gr...

Reviewer timeliness by area (based on a small sample of 14 papers * 4 reviewers each): Multilingual LMs >> VLMs > LLM Reasoning.

We have updated our collection of multilingual poetry corpora PoeTree. With addition of Norwegian, and few metadata fixes it now has a loud label of 1.0.0 release! 🌳 versologie.cz/poetree/vers...

Should be fun to read!

I can only go through <40 slides in a one-hour talk... verified multiple times. My job talk has 39 slides (including acknowledgement), and so are my recent talks. Always impressed when folks present 100 slides in the same amount of time.

In 2016 Hinton predicted that AI would replace all radiologists in five years. Ten years later, why hasn't it happened? This post is a great explainer. www.understandingai.org/p/ai-isnt-re...

🚀 ACL ARR is looking for a Co-CTO to join me lead our amazing tech team and drive the future of our workflow. If you’re interested or know someone who might be, let’s connect! RTs & recommendations appreciated.

Same! I‘ve literally unbidden 0 out of my recommended batch.

Is there a specific reason that NeurIPS does not show AC identity to reviewers? I'm very curious about who sent the polite reminders, and of course, even more curious about the rude ones.

#CoreCognition #LLM #multimodal #GrowAI We spent 3 years to curate 1503 classic experiments spanning 12 core concepts in human cognitive development and evaluated on 230 MLLMs with 11 different prompts for 5 times to get over 3.8 millions inference data points. A thread (1/n) - #ICML2025 ✅

Just in case this personal latex environmental setup is helpful to a broader crowd ⬇️

I'm late to the NAACL party, but I've just arrived in Albuquerque! I'll talk about measuring faithfulness of verbalized reasoning on *Sunday* at Repl4NLP at *9:45*.

hello NAACL friends I'm giving a keynote today at RepL4NLP at 1:30PM local time, come say hi! I'll mostly be musing about things with light research discussions

If you are also at NAACL, let's chat!

On my way to NAACL✈️! If you're also there and interested in grounding, don't miss our tutorial on "Learning Language through Grounding"! Mark your calendar: May 3rd, 14:00-17:30, Ballroom A. Another exciting collaboration with @marstin.bsky.social @kordjamshidi.bsky.social, Jiayuan, and Joyce!

📢Curious why your LLM behaves strangely after long SFT or DPO? We offer a fresh perspective—consider doing a "force analysis" on your model’s behavior. Check out our #ICLR2025 Oral paper: Learning Dynamics of LLM Finetuning! (0/12)

✈ Just landed in Singapore for #ICLR 2025! DM or email me if you'd like to chat about - Grounded language acquisition and learning - What do vision-language models "know," and what they don't - (Computational) linguistics with/for language models, especially grounded LMs (1/)

I received a review like this five years ago. It’s probably the right time now to share it with everyone who wrote or got random discouraging reviews from ICML/ACL.

I still can't comprehend how an AC (a professor) accepts the role but then never responds back to emails and never completes their tasks😖! We need urgent 5 emergency reviewers to complete reviews for ACL by the end of today. Area: Ethics, Bias, and Fairness. Please reach out if you can help! Thanks🙏

I was assigned to review three, and fortunately got a decent one. This is probably just the submission distribution nowadays.

🤖 I just made the slides public for this talk. TL; DR: how we computer scientists adapt insights from linguistics to analyze and improve our models. Comments & discussion are welcomed; the recording from Vector is forthcoming. docs.google.com/presentation...

There is a postdoctoral position for research on adversarial robustness at UBC, supervised by Mathias Lecuyer (@mathias-lecuyer.bsky.social), Geoff Pleiss, and Nidhi Hegde. Please spread the word! 🇨🇦 docs.google.com/forms/d/e/1F...

Customizing ggplot for yourself or your organization - slides and #RStats code from @pewresearch.org 's #NICAR25 presentation github.com/pewresearch/...

Another common suggestion I gave to almost all ICML papers I reviewed this time: whenever you write (CS/ML style) theory, please ensure all variables are defined. Although I can complete it with educated guess for most times, it's not always the case.

It's frustrating to see increasingly many false references in papers I'm reviewing. The authors write "claim about X (Y et al., 2019)" in the paper---as someone who has carefully read Y et al. (2019) and has done follow-up work, I'm confident there's no claim about X at all in it. #AmReviewing

The experiment I'm most excited about in this work is the disentanglement between language and domain. Even adding a semantically irrelevant sentence in another language to the demonstration---which just simply increases linguistic diversity---helps improve the performance!

Somewhat surprisingly, high-resource languages with non-Latin alphabets serve as better demonstrations than English. Increasing linguistic diversity in LLMs is not only about linguistic diversity itself---it's also about performance!

📜New analysis paper on the surprising effectiveness of multilingual chain of thought. Linguistic diversity in prompts helps solve problems in low-resource languages, even if there's no alphabetical overlap between the demonstrations and the target language.

[Update & Hiring] Last month, I joined Search and Recommendation Science team at Yahoo! as a Research Scientist🚀 Our team is hiring another Research Scientist🚀🚀 Send me a DM if you have strong research background in deep learning and NLP and want to be considered for the position🙋🙋🙋

ACL Rolling Review and the EMNLP PCs are seeking input on the current state of reviewing for *CL conferences. We would love to get your feedback on the current process and how it could be improved. To contribute your ideas and opinions, please follow this link! forms.office.com/r/P68uvwXYqfemn

This reminds me of the (rare) failure cases of training/finetuning NNs. Sometimes we got quite poor results without much reason. Restarting with another random seed magically works. Ideas of this work could be the key to understanding what's going on.

Ever looked at LLM skill emergence and thought 70B parameters was a magic number? Our new paper shows sudden breakthroughs are samples from bimodal performance distributions across seeds. Observed accuracy jumps abruptly while the underlying accuracy DISTRIBUTION changes slowly!

ARR February Authors: If you keep receiving confusing reminder emails about volunteer qualifications, please ask your nominated volunteer to update the OpenReview profile with either ACL Anthology or DBLP. This is how PCs verify qualification. RT appreciated!

I am looking for a postdoc. A serious-looking call coming soon, but this is to get it going. Topics include (but not limited to): LLMs (🫢!), multimodal LLMs, interaction+learning, RL, intersection with cogsci, ... see our work to get an idea: yoavartzi.com/pubs Plz RT 🙏

Excited that this got accepted at naacl/@naaclmeeting.bsky.social 2025! Massive kudos to Juan Diego and Aaron for being the best co-authors and colleagues one could ask for! 🙏

RepL4NLP (collocated with NAACL 2025) is inviting submissions!

We invite nominations to join the ACL2025 PC as reviewer or area chair(AC). Review process through ARR Feb cycle. Tentative timeline: Review 1-20 Mar 2025, Rebuttal is 26-31 Mar 2025. ACs must be available throughout the Feb cycle. Nominations by 20 Dec 2024: shorturl.at/TaUh9 #NLProc #ACL2025NLP

I will be at NeurIPS 2024 in Vancouver Dec 9-13. Please let me know if you'd like to learn more about r2llab.com or discuss potential collaborations. We will present 3 works around evaluating and building general-purpose language agents:

I'll be in Vancouver for NeurIPS from Dec 10 to 14. Happy to chat with people! Find me at - Dec 10: Future NLP Workshop with UBC NLP - Dec 12: Vector Luncheon - Dec 14: Pluralistic Alignment Workshop, presenting the VLM frame of reference analysis work with the amazing @marstin.bsky.social

I’m on the academic job market this year! I’m completing my @uwcse.bsky.social @uwnlp.bsky.social Ph.D. (2025), focusing on overcoming LLM limitations like hallucinations, by building new LMs. My Ph.D. work focuses on Retrieval-Augmented LMs to create more reliable AI systems 🧵

#rant #ARR Reviewer: Read this paper first. Me: Read my paper first. Or even better, use a better LLM.

First time submitting reference letters to MILA, and I LOVE the system: single button for uploading a pdf letter or click on "I decline to offer recommendation." Nothing else required, as all other information will be provided in letters. Other grad schools, please do consider learning from MILA.

If you claim your method offers 8% improvement over method X, based on a sample of size 12, I think you should consider taking a Statistic 101 class!

New thought after teaching CS486 Intro to AI at the beginning of my career, which might be obviously true to some people and completely wrong to others: - AI is about and only about planning.

easy first #booksky #nlp

( #NLP ) 😤

🚨New dataset + challenge🚨 We release ASL STEM Wiki: the first signing dataset of STEM articles! 📰 254 Wikipedia articles 📹 ~300 hours of ASL interpretations 👋 New task: automatic sign suggestion to make STEM education more accessible microsoft.com/en-us/resear... 🧵 #EMNLP2024

I'm sharing materials from my academic job search last year! Includes research, teaching, and diversity statements, plus my UMD cover letter and job talk slides. I applied for a mix of iSchool, data sci, CS, and linguistics positions). Feel free to share! juliamendelsohn.github.io/resources/

If you are interested in applying to grad school in ML/NLP, and are interested in our work in feedback learning, semantic evaluation, and test-time adaptation, please consider applying at r2llab.com/opening. DDL Dec 1! I'll also be presenting recent works on language agents at NeurIPS in Vancouver.