This was accepted to #NeurIPS 🎉🎊 TL;DR Impoverished notions of rigor can have a formative impact on AI work. We argue for a broader conception of what rigorous work should entail & go beyond methodological issues to include epistemic, normative, conceptual, reporting & interpretative considerations

We are excited to kick off the 2nd HEAL workshop tomorrow at #CHI2025. Dr. Su Lin Blodgett and Dr. Gagan Bansal from MSR will be our keynote speakers! Welcome new and old friends! See you at G221! All accepted papers: tinyurl.com/bdfpjcr4

Bringing together our incredible current and admitted students—future leaders, innovators, and changemakers!

📣 DEADLINE EXTENSION 📣 By popular request, HEAL workshop submission deadline is extended to Feb 24 AOE! Reminder that we welcome a wide range of submissions: position papers, lit reviews, encore of published work, etc. Looking forward to your submissions!

Thrilled that our paper Faux Polyglot has been accepted to #NAACL2025 main! 🚀 We show that multilingual RAG creates language-specific information cocoons and amplifies perspectives and facts in the dominant language, especially when handling knowledge conflicts. 📜 arxiv.org/abs/2407.05502

The submission deadline is in less than a month! We welcome encore submissions, so consider submitting your work regardless of whether it's been accepted or not #chi2025 😉

Human-centered Evalulation and Auditing of Language models (HEAL) workshop is back for #CHI2025, with this year's special theme: “Mind the Context”! Come join us on this bridge between #HCI and #NLProc! Workshop submission deadline: Feb 17 AoE More info at heal-workshop.github.io.

Super excited to announce that @msftresearch.bsky.social's FATE group, Sociotechnical Alignment Center, and friends have several workshop papers at next week's @neuripsconf.bsky.social. A short thread about (some of) these papers below... #NeurIPS2024

📣 📣 Interested in an internship on human-centred AI, human agency, AI evaluation & the impacts of AI systems? Our team/FATE MLT (Su Lin Blodgett, @qveraliao.bsky.social & I) is looking for a few summer interns 🎉 Apply by Jan 10 for full consideration: jobs.careers.microsoft.com/global/en/jo...

Seeing cool works on metrology and measurement modeling for NLP! So I wanted to port over the thread our ACL 2023 Findings paper (arxiv.org/abs/2305.09022) on conceptualizations of NLP tasks and measurements of performance! Work with Eric Yuan, @haldaume3.bsky.social, and Su Lin Blodgett. (1/n)

I am collecting examples of the most thoughtful writing about generative AI published in 2024. What’s yours? They can be insightful for commentary, smart critique, or just because it shifted the conversation. I’ll post some of mine below as I go through them. #criticalAI

Created a small starter pack including folks whose work I believe contributes to more rigorous and grounded AI research -- I'll grow this slowly and likely move it to a list at some point :) go.bsky.app/P86UbQw

Hi, so I've spent the past almost-decade studying research uses of public social media data, like e.g. ML researchers using content from Twitter, Reddit, and Mastodon. Anyway, buckle up this is about to be a VERY long thread with lots of thoughts and links to papers. 🧵

Had a lot of fun teaching a tutorial on Human-Centered Evaluation of Language Technologies at #EMNLP2024, w/ @ziangxiao.bsky.social, Su Lin Blodgett, and Jackie Cheung We just posted the slides on our tutorial website: human-centered-eval.github.io

🚨 NeurIPS 2024 Spotlight Did you know we lack standards for AI benchmarks, despite their role in tracking progress, comparing models, and shaping policy? 🤯 Enter BetterBench–our framework with 46 criteria to assess benchmark quality: betterbench.stanford.edu 1/x

It turns out we had even more papers at EMNLP! Let's complete the list with three more🧵

Our lab members recently presented 3 papers at @emnlpmeeting.bsky.social in Miami ☀️ 📜 From interpretability to bias/fairness and cultural understanding -> 🧵

my first post, now that I am here with my 500+ closest friends 🙂 -- here is a tiny owl 🦉 I met some weeks back in the big apple 🍎 (picture by @sbucur.bsky.social)

McGill NLP just landed on this blue planet bsky.app/profile/mcgi...

The starter pack just surpassed 1/3 of its capacity! Don't be shy to reach out to me if you are a researcher in this area, or if you have suggestions. Thank you 🥰

I didn’t expect to wind up in the news over this but in hindsight, I guess it makes sense lol. This is the first time I’ve been in the Herald since high school 😂.

“We argue that societal impacts [of GenAI] should be conceptualised as application- and context-specific, incommensurable, and shaped by questions of social power.” By @glenberman.bsky.social et al. arxiv.org/abs/2410.22985

I’m putting together a starter pack for researchers working on human-centered AI evaluation. Reply or DM me if you’d like to be added, or if you have suggestions! Thank you! (It looks NLP-centric at the moment, but that’s due to the current limits of my own knowledge 🙈) go.bsky.app/G3w9LpE

Putting together a JHU Center for Language and Speech Processing starter pack! Please reply or DM me if you're doing research at CLSP and would like to be added - I'm still trying to find out which of us are on here so far. go.bsky.app/JtWKca2

If you are at #EMNLP2024, don't miss the "Human-Centered Evaluation of Language Technologies" tutorial on Saturday 14:00-17:30! My awesome mentors will be teaching about this awesome topic ❤️

Shameless plug: a post on 'writing as thinking' that I wrote for my own students hackingsemantics.xyz/2024/writing/

Hello, Computational linguistics/NLP world in Bluesky! We're creating the same accounts on other social media platforms in Bluesky! #NLProc

🚚 Moving threads about my #nlp papers from Twitter to here 🚚 Do NLP benchmark measurements provide meaningful insights about the evaluated models? To help practitioners answer these questions, we introduce ECBD - a conceptual framework that formalizes the process of benchmark design 🧵.

🚚 Moving threads about my #nlp papers from Twitter to here 🚚 How and when, and with which issues, does the text summarization community engage with responsible AI? 🤔 In this #EMNLP2023 paper, we examine reporting and research practices across 300 summarization papers published between 2020-2022 🧵