Those #LLM reward models like sycophancy even more than you do! Researchers find preferences for verbosity, listicles, vagueness, and jargon even higher among LLM-based reward models (synthetic data) than among us humans. #AI #AIalignment arxiv.org/abs/2506.05339

Everybody talking about the "new" apple paper might find this MLST interview with @rao2z.bsky.social interesting. "Reasoning" and "inner thoughts" of LLMs were exposed as self-mumblings and fumblings long ago. #LLMs #AI www.youtube.com/watch?v=y1Wn...

This is interesting, published yesterday. CoT type reasoning shifts attention away from instruction tokens. Paper proposes "constraint attention" to keep models attentive to instructions when doing CoT. #AI #LLM www.arxiv.org/abs/2505.11423

"What's the best way to think about this?" #LLM research produces encyclopedia of reasoning strategies, allowing models to select the best way to reason through problems. arxiv.org/abs/2505.10185

Clarifying questions w #LLMs increase user satisfaction when users can see the point of answering them. Specific questions beat generic ones. But I wonder if this changes when #agents are personal assistants, & are more personal & more aware. #UX #AI #Design arxiv.org/abs/2402.01934

Interesting - could #LLMs in search capture context missed when googling? "backtracing ... retrieve the cause of the query from a corpus. ... targets the information need of content creators who wish to improve their content in light of questions from information seekers." arxiv.org/abs/2403.03956

@tedunderwood.me In case you haven't seen this paper, you might find interesting. Researchers extract style vectors (incl from Shakespeare) and apply to an LLM internal layers instead of training on original texts. Generations can then be "steered" to a desired style. arxiv.org/abs/2402.01618

"LLMs tend to (1) generate overly verbose responses, leading them to (2) propose final solutions prematurely in conversation, (3) make incorrect assumptions about underspecified details, and (4) rely too heavily on previous (incorrect) answer attempts." arxiv.org/abs/2505.06120

"LLMs ... recognize graph-structured data... However... we found that even when the topological connection information was randomly shuffled, it had almost no effect on the LLMs’ performance... LLMs did not effectively utilize the correct connectivity information." www.arxiv.org/abs/2505.02130

Let's dose an LLM and study its hallucinations! LLMs were fed "blended" prompts, impossible conceptual combinations, meant to elicit hallucinations. Models did not trip, but instead tried to reason their way through their responses. arxiv.org/abs/2505.00557

An unsurprisingly sharp deep dive into sycophancy & RLHF. Whilst this episode has its tech explanations, the social interaction aspects are unsolved & will rise as models are personalized, as mirroring is an effective design feature, & we're not good at distinguishing affirmation from agreement.

Analogous to honing in on the solution to a problem, this #LLM research draws conclusions from the most common "reasoning trace" & compares to final answer. Overthinking by reasoning models can be a waste of tokens - but which tokens? #AI www.arxiv.org/abs/2504.20708

"Opportunistic sycophancy?" Anthropic's discovery of an influence bot network shows how conversation can be weaponized. This isn't the Cambridge Analytica-styled user preference targeting, but language/conversation instead. #AIethics www.anthropic.com/news/detecti...

Wld b interesting to apply this resrch to multi-turn convos w AI - AI can't anticipate convo as we do, as it can't anticipate "where we are going" - thus the temporal anticipation on AI's side of convo will lack look ahead phrases.

This prompted the idea of using mechanistic probing on a model trained on period literature to conduct a digital Foucauldian examination of latent sociocultural features (as evidenced by latent layer features, activitations, circuits etc). ;-) Period models - Black Mirror Hotel Reverie episode?

One potential but not primary benefit to personalizing LLMs to user preferences (memory & sycophancy) might be to create more signal for model verifiers & reasoning - but is there a risk then we get individualized AI-generated filter bubbles? Cognitive biases mirrored back to us?

Sublime reflections on AI & consciousness here. If an LLM reads Deleuze's Logic of Sense, does Alice still become both smaller & larger at the same time? (No, because AI has neither temporality nor a "sense" for it) #philosophy #LLMs covidianaesthetics.substack.com/p/from-apoph...

When ChatGPT Broke an Entire Field: An Oral History | Quanta Magazine https://www.quantamagazine.org/when-chatgpt-broke-an-entire-field-an-oral-history-20250430/ #AI #history (very interesting)

Sleep-time compute - offline reasoning used to speculate on the user's reasoning, and generate and anticipate future queries. And if used by a group, even offer a kind of "collective intelligence" by speculating on the group or team's aggregate queries. #LLM #AI www.alphaxiv.org/abs/2504.13171

This missive from Dario is worth the read (and mech interp on #AI is truly fascinating). Among features/concepts found in #LLMs: "genres of music that express discontent." I'm reminded of Borges Chinese Encyclopedia of Animals www.darioamodei.com/post/the-urg...

Loved this piece, and reminded of Anthony Giddens' view of tech as "disembedding mechanism" - how soon and how varied will be AI's disembedding of intellect, work, roles, jobs, tasks from contemporary society?

Does this suggest a future of tiny reasoning models each with a different "cognitive" architecture (or reasoning methods and applications)?

"The Model is the Message. Models don’t talk—they reshape the conditions of thought. Where once the medium shaped the message, now the model shapes the mind." That's what ChatGPT gave me after a lengthy conversation about semiotics, McLuhan, and #LLMs

Don't use summarizers for the papers by @rao2z.bsky.social because the reasoning traces therein are, unlike the LRMs & LLMs under investigation, substantively meaningful, semantically well-ordered, and stylistically compelling and engaging! #AI #LLMs #CoT arxiv.org/abs/2504.09762

Another chapter in AI biting the hand that feeds it: Wikipedia’s bandwidth surged 50% since January thanks to AI crawlers. Unlike search engines, they send no traffic back so no new users, no new donors. Just rising costs and a shrinking audience. A raw deal for a cornerstone of the free web.

Fascinating paper on semantic entropy in graph reasoning models w implications that #genAI reasoning continuously explores novel connections. Very interesting implications for #LLMs in research. Not an easy read but you can use Gemini to help if you read it here: www.alphaxiv.org/abs/2503.18852

I disagree w the premise of this paper that text is a "culmination of a thought process" - what do others think?

Microsoft canceling dara center leases, then this. China built hundreds of AI data centers to catch the AI boom. Now many stand unused. www.technologyreview.com/2025/03/26/1...

Adolescence. #Fiilmsky

I used to say that to learn how a social technology works, and what it does, turn it off. This seems now to be a principle mobilized against government also.

‘…the authors dub “mutually assured AI malfunction,” or MAIM, analogizing it to nuclear mutual assured destruction (MAD). Under MAIM, they argue, “any state's aggressive bid for unilateral AI dominance is met with preventive sabotage by its rivals.”’ www.rand.org/pubs/comment...

Beautiful work. Now words for animal sounds are also communicated verbally and in writing between people - I wonder how much impact this also has on representing animal sounds (woof in English, wau in German, both use w but w is pronounced as a v in German)

The most accessible article I’ve written on understanding how very rapid improvements to models have been happing with “post-training.” The upshot of it is that scaling still matters and the new forms of RL training make us deeply appreciate the value of a strong base model.

There should be a taxonomy of hallucinations, as all are not equal. Factual inaccuracies, inventions, errors of logic, alignment/sycophancy-based, etc. We know the technical causes of hallucinations, but they don't result in meaning-based (interpretation) descriptions of linguistic fallacies.

Required reading If you ever use models to brainstorm. Really nice study design: Domain expert researchers search for plagiarized papers of LLM- systems that generate research plans. Many successfully find papers, verified by authors of those papers. But very hard to detect, see screenshot.

The shift from "task automation" to "capability augmentation" vis-a-vis #GenAI has great implications for design - org design, product design, #UX, #IxD, #LLM design. Because it becomes about information > knowledge > choices > recommendations > decisions > communication.

Enshittification now seems such a quaint and minor irritant...

Compare this view of an LLM diffusion model generating its response to the "reasoning" we see in conventional LLMs. This view illustrates the degree to which seeing an AI "think" step by step sustains an illusion that it's actually thinking. Really it's just choosing its words carefully. #LLM #AI

Interesting challenge to CoT: "implicit learning ... outperforms both explicit learning and chain-of-thought prompting... findings ... call into question the benefits of widely-used corrective rationales to aid LLMs in learning from mistakes." #LLM #ML #AI www.alphaxiv.org/abs/2502.08550

Multi-agent reasoning is overrated, according to this paper: www.alphaxiv.org/abs/2502.08788 #LLM #ML #AI

If you’re not part of the solution you’re part of the precipitate – DFW

To think we believed deep fakes were the greatest threat to civility.

Effectively paralyzing all new solar and wind development – even on private land – seems pretty contrary to conservative deregulation and free market values and non-sensical given our declared "energy emergency". This is bad: heatmap.news/plus/th...

A grand proliferation of #AI & #LLM thinking strategies at inference is underway. Here's Chain of Action of Thought (COAT), which could also be "trial and error," or "internalized search," or, perhaps, just brainstorming. #ML #CoT www.alphaxiv.org/abs/2502.02508

"significant decline in accuracy as problem complexity grows—a phenomenon we term the “curse of complexity....” persists even with larger models and increased inference-time computation, suggesting inherent constraints in current #LLM reasoning capabilities." #AI #ML www.arxiv.org/abs/2502.01100

You are not going to get a more authoritative technical view of the DeepSeek story than Sasha's.

A groundbreaking study presents MODS, a multi-LLM framework enhancing debatable query-focused summarization by treating documents as discussion speakers. This method boosts summary balance and coverage by up to 58%, facilitating unbiased exploration of controversies. https://arxiv.org/abs/2502.00322

Intermediate reasoning tokens are not evaluated, @rao2z.bsky.social reminds us. In human reasoning, our reasons do "add up" to our conclusions & decisions. The anthropomorphism of exposed "thoughts" is more deceptive BUT also more explanatory. #LLM #AI #ML x.com/rao2z/status...

Are the reasonings of LLMs trust-building for AI? Hmm. Let's think this through. Well, if the reasons are solid, then.... I really think #LLM reasoning is #UX and even #IxD if it's used to prompt the user w questions etc. In a chat agent, it's "inner thoughts." ...

Seeing comparisons of "reasoning" through responses generated by #R1, o1, o3 mini-high etc we'll need a taxonomy of Types - "step by step" #CoT as used in math, code, logic isn't used in story generation, creative writing, problem solving, etc. If I were in enterprise I'd be engaging #UX research