Linguistics PhD student @jacksonpetty.org finds LLMs "quiet-quit" when instructions get long, switching from reasoning to guesswork. With CDS' @tallinzen.bsky.social, @shauli.bsky.social, @lambdaviking.bsky.social, @michahu.bsky.social, and Wentao Wang. nyudatascience.medium.com/llms-switch-...

you’re telling me a star spangled this banner??

I'm hiring at least one post-doc! We're interested in creating language models that process language more like humans than mainstream LLMs do, through architectural modifications and interpretability-style steering. Express interest here: docs.google.com/forms/d/e/1F...

How well can LLMs understand tasks with complex sets of instructions? We investigate through the lens of RELIC: REcognizing (formal) Languages In-Context, finding a significant overhang between what LLMs are able to do theoretically and how well they put this into practice.

[guy who only watched the pilot episode of Caprica voice] Nice!

Looking forward to speaking tomorrow (Tues am) in this Simons workshop in Berkeley simons.berkeley.edu/workshops/ll.... Will talk about some empirical work and also share some takes from this recent preprint from me and @futrell.bsky.social arxiv.org/abs/2501.17047

@foodnoms.com Can I submit a feature request on bsky? It would be amazing if we could tie Goals to bodyweight, as read in from Apple Health data. Many recommendations for daily protein intake, for instance, are given in grams/kg of bodyweight.

Eighth night // Dedication of the House (Shimen Frug)

Seventh night // Dedication of the House (Shimen Frug)

Happy Hanukkah

shabbat shalom

בימים ההם בזמן הזה

Accidentally bought tickets to the Sing Along version of Nosferatu 😑

Happy Hanukkah

Later on, we'll compile, As we train on The Pile, To autoregress The next words we'll guess Grokking in the latent feature land

I think 2m43s is a new personal record

o3, o3 in the loop, list the steps in making soup

Well one couldn't help but become mildly sensitive to the context over there!

“You’re telling me a Tree Adjoined this Grammar??” — @bob-frank.bsky.social

Claude understood the assignment (a bit too well)

I get why conservatives dislike much of academia, however even taking into account academia's excesses, I resent their parochial approach because it disregards at a fundamental level what academia pretends: that everything is worthy of analysis.

URL verification works fine. The bigger issue is that people are reluctant to use a URL as their handle without the ability to reserve the *.bsky.social version and have it redirect

Of all sad words of tongue or pen, the saddest are these:

People who regularly use emojis, please choose the following sentence which is least acceptable. 1️⃣ 🫵 said that 🫵 would help me! 2️⃣ 🫵 said that you would help me! 3️⃣ You said that 🫵 would help me! 4️⃣ I have no judgments on this 📊 Show results

girlfriend tax implies the existence of the girlfriend laugher curve, which is the theoretical relationship between how bad my jokes are and how much she laughs at them

girlfriend tax implies the existence of girlfriend industrial policy

home economics, but I repeat myself

Barbara Partee ‘Noun Phrase Interpretation and Type-shifting Principles’ (1987) // Sir Harris Nicolas ‘Aigncourt banner of the Trinity’ (1833)

1) what

polymarket for which track my train to boston is leaving penn station on

so apparently the Ancient Greek verb for “to hurt” is just “aaaaaooooow”

You should go work with Isabel

In case it’s of use to other academics, I put together an overleaf project which shows how to use different styles for in-text citations and bibliographies (eg, author-year in text vs numeric in bib). Works in BibLaTeX and hackily in natbib. Link: www.overleaf.com/read/tkxbfty...

Pingali and Bilardi (2015) just get me

A highlight of EMNLP: finding a fellow Heliotrope enthusiast in @venkatasg.net

Scenes from Miami

🤖🧠 I'll be considering applications for postdocs & PhD students to start at Yale in Fall 2025! If you are interested in the intersection of linguistics, cognitive science, and AI, I encourage you to apply! Postdoc link: rtmccoy.com/prospective_... PhD link: rtmccoy.com/prospective_...

November in New Haven

“Something is ROT-10 in the state of Denmark” slow clap, @rtommccoy.bsky.social, slow clap

if you’re at EMNLP and wanna brainstorm ideas around: - LLMs for ancient languages - code & formal languages for LLMs - applying (😱) math to better understand LLMs we should chat this week!

Are you coming to #EMNLP2024? Come say hi! I’ll be presenting this work at BlackboxNLP on Friday, and will be around all week to chat! DM to meet up, let’s grab lunch!

Our paper on how model depth impacts compositional generalization was accepted to #NAACL2024 🇲🇽 A new version is up on arXiv! We've added a discussion of how making models deeper makes them slower, without adding much benefit for compositional tasks arxiv.org/abs/2310.19956

How well do LLMs generalize to novel distributions? Our (Michael Wilson Bob Frank) new TACL / EMNLP 2023 paper uses argument structure to test this, finding that LLMs succeed within known contexts but fail to do so between appropriately related ones. 🧵 jacksonpetty.org/argstruct

Do deep transformer LMs generalize better? In a new preprint we (Sjoerd van Steenkiste, Ishita Dasgupta, Fei Sha, Dan Garrette, & @tallinzen.bsky.social) control for parameter count to show how depth helps models on compositional generalization tasks, but diminishingly so 🧵 jacksonpetty.org/depth

20-year-old Harvard students are perfectly capable of thinking through the consequences of supporting terrorism, and are therefore entirely responsible the consequences of someone screenshotting what they did and attaching their names to it

We now turn to live coverage of the @nytimes.com editors room