Our forthcoming NeurIPS position paper, led by @aolteanu.bsky.social, makes this argument (along with several related ones) in more depth. Rigorous AI/ML work should flow from explicit and rigorous premises, not just have a final evaluation that checks some rigor boxes. arxiv.org/abs/2506.14652

I wish folks would use more precise terminology than "AI sycophancy." Not all validating behaviours/interactions are sycophantic. By definition, for them to be sycophantic there needs to be an underlying intention to e.g., gain advantage or favour. Intention is something AI systems do not have.

Love this analogy

This was accepted to #NeurIPS 🎉🎊 TL;DR Impoverished notions of rigor can have a formative impact on AI work. We argue for a broader conception of what rigorous work should entail & go beyond methodological issues to include epistemic, normative, conceptual, reporting & interpretative considerations

"Epistemic rigor, however, does not necessarily require specific epistemological commitments or choices but rather that those commitments and choices be made explicit." www.arxiv.org/abs/2506.14652

Listening to a workshop panel at #acl2025 I am realizing that we are saying more or less the same things and having more or less the same conversations for so many years

#acl2025 I think there is plenty of evidence for the risks of anthropomorphic AI behavior and design (re: keynote) -- find @myra.bsky.social and I if you want to chat more about this or our "Dehumanizing Machines" ACL 2025 paper

In a stunning moment of self-delusion, the Wall Street Journal headline writers admitted that they don't know how LLM chatbots work.

Who is attending @aclmeeting.bsky.social in Vienna? Reach out or find me there if you want to chat! #acl2025nlp

My university has announced a fund to essentially poach doctoral students from US institutions. DM me if you do work on the history/social impacts of AI and are interested in being poached 😂

Not sure who needs to hear this but what people want AI systems to do, what AI systems do, and what people believe AI systems do are not the same thing. Just because one wants or believes AI systems do or can do certain things, doesn't mean they actually do those things.

If you're at @icmlconf.bsky.social this week, come check out our poster on "Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge" presented by the amazing @afedercooper.bsky.social from 11:30am--1:30pm PDT on Weds!!! icml.cc/virtual/2025...

Do you have strong programming skills but need research experience doing meaningful & exciting CSS projects before heading off to a top graduate school for computational social science PhD? Apply now to predoc with me, @dggoldst.bsky.social @jakehofman.bsky.social www.microsoft.com/en-us/resear...

Someone asked me today how to get better at scientific writing. I'm not the best person to ask because I find my own writing very inadequate! But the tips I thought of were: 1. Practice, and practice with co-authors who are better writers than you. Observe how they make edits and copy them. (1/n)

FAccT is such a special community & many of us have invested a lot of service time/effort to support it over the years. I do believe engaging with uncomfortable questions & dialogue is important even when there is criticism (which can be hard to hear, can feel unfair/demotivating & sucks) #facct2025

Flattered and shocked for our paper to receive the #facct2025 best paper award.

Two years after the craft session on theories of change in responsible AI, I am glad to see this discussion taking central stage as a keynote panel #facct2025

There is a lot of talk and effort to figure out how genAI is different (I am also guilty of this!) -- the reality is that genAI is not that different and genAI is not that new either; it was hard to evaluate in the past, and it is still as hard to evaluate now #facct2025

Your #FAccT2025 General Chairs @sciorestis.bsky.social, @metaxa.net, and I, reporting from the venue. We're looking forward to welcoming you to the Athens Conservatoire or online!

Who is going to @facct.bsky.social? I will be arriving in Athens tomorrow late morning and looking forward to catching up with old and new friends at FAccT ☀️

We have to talk about rigor in AI work and what it should entail. The reality is that impoverished notions of rigor do not only lead to some one-off undesirable outcomes but can have a deeply formative impact on the scientific integrity and quality of both AI research and practice 1/

Alright, people, let's be honest: GenAI systems are everywhere, and figuring out whether they're any good is a total mess. Should we use them? Where? How? Do they need a total overhaul? (1/6)

I'm so excited this paper is finally online!!! 🎉 We had so much fun working on this with @emmharv.bsky.social!!! Thread below summarizing our contributions...

📣 "Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems" is forthcoming at #ACL2025NLP - and you can read it now on arXiv! 🔗: arxiv.org/pdf/2506.04482 🧵: ⬇️

Exciting news!!! This just got into @icmlconf.bsky.social as a position paper!!! 🎉 More updates to come as we work on the camera-ready version!!!

I'm joining Wisconsin CS as an assistant professor in fall 2026!! There, I'll continue working on language models, computational social science, & responsible AI. 🌲🧀🚣🏻‍♀️ Apply to be my PhD student! Before then, I'll postdoc for a year in the NLP group at another UW 🏔️ in the Pacific Northwest

Life update! Excited to announce that I’ll be starting as an assistant professor at Cornell Info Sci in August 2026! I’ll be recruiting students this upcoming cycle! An abundance of thanks to all my mentors and friends who helped make this possible!!

This draws on work from last summer at MSR MTL, with @aolteanu.bsky.social, Su Lin Blodgett, @uhleeeeeeeshuh.bsky.social, and @lisaegede.bsky.social! Check out the full post: iclr-blogposts.github.io/2025/blog/an...

New ICLR blogpost! 🎉 We argue that understanding the impact of anthropomorphic AI is critical to understanding the impact of AI.

Presenting this at #CHI2025 tomorrow, Monday, April 28 in the "Expressive Machines" (lol 🤷‍♀️) session at 4:44 p.m. in Annex Hall F206

I don't know about others, but I am just don't understand people who don't like em dashes. I just don't get it.

Is everyone now okay with using the term "thinking" to describe what LLM "reasoning" models do? And to call their outputs "thoughts"? From OpenAI blog posts:

Great work from an amazing team! 🌟 Check out this thread to learn more about our work on anthropomorphic AI systems:

How can we better think and talk about human-like qualities attributed to language technologies like LLMs? In our #CHI2025 paper, we taxonomize how text outputs from cases of user interactions with language technologies can contribute to anthropomorphism. arxiv.org/abs/2502.09870 1/n

Check out our recent work studying anthropormophic AI!

Our FATE MTL team has been working on a series of projects on anthropomorphic AI systems for which we recently put out a few pre-prints I’m excited about. While working on these we tried to think carefully not only about key research questions but also how we study and write about these systems

Remember this @neuripsconf.bsky.social workshop paper? We spent the past month writing a newer, better, longer version!!! You can find it online here: arxiv.org/abs/2502.00561

Bumping this up 🔉 If interested in interning with me or my colleagues apply by Friday, Jan 10 for full consideration! We are especially looking for candidates interested responsible and ethical AI considerations related to human agency, human control, anthropomorphic AI systems, and measurement

One of the activities I like the least (though something I do almost daily) is searching for my glasses without my glasses.

I'll be giving a short talk on "Evaluating Generative AI Systems is a Social Science Measurement Challenge" (arxiv.org/abs/2411.10939) at 230pm today at the @msftresearch.bsky.social booth! #NeurIPS2024

New preprint, w/ @lilianedwards.bsky.social: 'Reputation Management in the #ChatGPT Era': www.reubenbinns.com/blog/wp-cont... We consider what legal tools can protect individuals when #GenAI outputs lead to reputational /privacy harms with a particular focus on defamation and data protection law.

I'm hiring PhD students at Brown CS! If you're interested in human-robot or AI interaction, human-centered reinforcement learning, and/or AI policy, please apply. Or get your students to apply :). Deadline Dec 15, link: cs.brown.edu/degrees/doct.... Research statement on my website, www.slbooth.com

Question: Is there any other example than 2024 FAccT (and I believe AIES) of a computing conference which similarly encourages authors to include positionality statements?

Evaluating Generative AI Systems is a Social Science Measurement Challenge: arxiv.org/abs/2411.10939 TL;DR: The ML community would benefit from learning from and drawing on the social sciences when evaluating GenAI systems.

Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems: arxiv.org/abs/2411.15662 TL;DR: We identify challenges to using publicly available measurement instruments for measuring representational harms. Led by our intern @emmharv.bsky.social.

That being said, I don't think of myself as an "AI skeptic", but as a scientist. The *job* of scientists is to be skeptical of big claims & to test them out (or to encourage more careful testing of them). (2/2)

Question: Is there any other example than 2024 FAccT (and I believe AIES) of a computing conference which similarly encourages authors to include positionality statements?

It is that time of year again we are looking for summer 2025 interns at FATE Montreal. Apply!