Philip K. Dick, "The Penultimate Truth", Belmont Books, 1964.

Rikiya Takehi, Fernando Diaz, Tetsuya Sakai Diversification as Risk Minimization https://arxiv.org/abs/2510.22681

📢 Announcing the First Workshop on Multilingual and Multicultural Evaluation (MME) at #EACL2026 🇲🇦 MME focuses on resources, metrics & methodologies for evaluating multilingual systems! multilingual-multicultural-evaluation.github.io 📅 Workshop Mar 24–29, 2026 🗓️ Submit by Dec 19, 2025

We’re excited to release the Call for Papers for #FAccT2026 which will be held in Montreal, Canada in June 2026! Abstracts are due on January 8th, papers due on January 13th. Call for Papers: facctconference.org/2026/cfp Important info in thread →

AI is evolving too quickly for an annual report to suffice. To help policymakers keep pace, we're introducing the first Key Update to the International AI Safety Report. 🧵⬇️ (1/10)

I am on the job market this year! My research advances methods for reliable machine learning from real-world data, with a focus on healthcare. Happy to chat if this is of interest to you or your department/team.

Renee Shelby, Fernando Diaz, Vinodkumar Prabhakaran: Taxonomy of User Needs and Actions https://arxiv.org/abs/2510.06124 https://arxiv.org/pdf/2510.06124 https://arxiv.org/html/2510.06124

MSR NYC is hiring spring and summer interns in AI/ML/RL! Apply here: jobs.careers.microsoft.com/global/en/jo...

In January, Asia Biega (MPI), Georgina Born (UCL), Mary Gray (MSR), Rida Qadri (G), and I ran a Dagstuhl Seminar bringing together folks from CS and the broader social sciences to discuss questions around AI and culture. Dagstuhl has just posted our report, 1/3 drops.dagstuhl.de/storage/04da...

This was accepted to #NeurIPS 🎉🎊 TL;DR Impoverished notions of rigor can have a formative impact on AI work. We argue for a broader conception of what rigorous work should entail & go beyond methodological issues to include epistemic, normative, conceptual, reporting & interpretative considerations

I'm noticing that some conversational AI interfaces are pausing text generation (or at least rendering) until the user scrolls. A thread on attention modeling. 👀 🧵 1/10

🚨Microsoft Research NYC is hiring🚨 We're hiring postdocs and senior researchers in AI/ML broadly, and in specific areas like test-time scaling and science of DL. Postdoc applications due Oct 22, 2025. Senior researcher applications considered on a rolling basis. Links to apply: aka.ms/msrnyc-jobs

Giuliano, V.E. and P. E. Jones, "Linear Associative Information Retrieval", Rept. no. CACL-2, Arthur D. Little, Inc., Cambridge, Mass., Nov 1962, 240 p. Also in P. W. Howerton and D. C . Weeks [eds ]. "Vistas in Information Handling", 1963, p. 30-54.

junior faculty in STSy positions, check out the CIFAR global scholars call: cifar.ca/next-generat... I'm on the "Future Flourishing Program," and it's a nice community (and a good bit of money as well).

My university has announced a fund to essentially poach doctoral students from US institutions. DM me if you do work on the history/social impacts of AI and are interested in being poached 😂

To Eun Kim just presented the work on "Tip of the Tongue Query Elicitation for Simulated Evaluation" at #SIGIR2025. The approach will be used in the #TREC2025 Tip-of-the-Tongue track, and we had some sweets at the poster :) The paper is available online: dl.acm.org/doi/10.1145/...

Lukas Gienapp presents "The Viability of Crowdsourcing for RAG Evaluation" at #SIGIR2025 The paper is available at: webis.de/publications...

Hello TREC-ToTers! We have released the test queries for the TREC 2025 Tip-of-the-Tongue (TREC-ToT) Track. Please see the guidelines for more information: trec-tot.github.io/guidelines. Run submission deadline will tentatively be in August. #TREC2025 #TRECToT #TREC2025ToT Please spread the word!

I've always felt that Montréal is a safe space for research -- it’s a city that values science, collaboration, and diversity, and I’m proud to call it home 🏠 Like I said in the video: Montréal is home—and all are welcome.💖🦋 youtu.be/dWiL4b7-KEA

We're happy to officially announce the location of #FAccT2026! Next year's conference will be held in Montreal, Canada 🇨🇦 Su Lin Blodgett and Zeerak Talat will be General Chairs, and Michael Madaio will be PC Chair 🎉 (thanks to MindView for the photo!)

p epic conclusion in the og description of the sign test. John Arbuthnot. An argument for divine providence, taken from the constant regularity observ'd in the births of both sexes. Philosophical Transactions of the Royal Society of London, 27(328):186-190, 1710.

There is a lot of talk and effort to figure out how genAI is different (I am also guilty of this!) -- the reality is that genAI is not that different and genAI is not that new either; it was hard to evaluate in the past, and it is still as hard to evaluate now #facct2025

We have to talk about rigor in AI work and what it should entail. The reality is that impoverished notions of rigor do not only lead to some one-off undesirable outcomes but can have a deeply formative impact on the scientific integrity and quality of both AI research and practice 1/

When it comes to text prediction, where does one LM outperform another? If you've ever worked on LM evals, you know this question is a lot more complex than it seems. In our new #acl2025 paper, we developed a method to find fine-grained differences between LMs: 🧵1/9

🖋️ Curious how writing differs across (research) cultures? 🚩 Tired of “cultural” evals that don't consult people? We engaged with interdisciplinary researchers to identify & measure ✨cultural norms✨in scientific writing, and show that❗LLMs flatten them❗ 📜 arxiv.org/abs/2506.00784 [1/11]

Many libraries now use automated tools to measure diversity in their collections. We examined how these tools work and whether library workers find them useful. A complex case study of libraries navigating automation, DEI, & shrinking public funding. Our new FAccT paper: arxiv.org/abs/2505.14890

Exciting news: The Fairness, Accountability, Transparency and Ethics (FATE) group at Microsoft Research NYC is hiring a predoctoral fellow!!! 🎉 www.microsoft.com/en-us/resear...

The Fairness, Accountability, Transparency, and Ethics (FATE) research group at Microsoft Research New York City (MSR NYC) is looking for a pre-doctoral research assistant! www.microsoft.com/en-us/resear...

vignette from, Paul R. Cohen. Empirical methods for artificial intelligence. MIT Press, 1995. 1/3

Who would make you really excited if they joined Ai2? We always are looking to hire these people that seem like obvious strong fits.

possibly better, imps for christ

Ever trusted a metric that works great on average, only for it to fail in your specific use case? In our #NAACL2025 paper (w/ @841io.bsky.social), we show why global evaluations are not enough and why context matters more than you think. 📄 aclanthology.org/2025.finding... #NLP #Evaluation (🧵1/9)

If you're interested in OpenAI including shopping results, you might also be interested in @teknology.bsky.social's paper relating retrieval diversity/fairness and generation by downstream RAG models. This has implications for individuals selling products online. arxiv.org/abs/2409.11598

this morning while reading up on the lineage of the Wasserstein distance... Toom, A. (1993). A Russian Teacher in America. The Journal of Mathematical Behavior, 12(2).

📢 New Paper: "Recall, Robustness, and Lexicographic Evaluation" (ACM TORS) F Diaz, M Ekstrand (@md.ekstrandom.net), B Mitra (@bmitra.bsky.social) For IR, NLP, and ML researchers working on ranking systems evaluated for recall and robustness. 🧵 1/5 dl.acm.org/doi/10.1145/...

🚨New Breakthrough in Tip-of-the-Tongue (TOT) Retrieval Research! We address data limitations and offer a fresh evaluation method for these complex queries. Curious how TREC TOT track test queries are created? Check out this thread 🧵 and our paper 📄: arxiv.org/abs/2502.17776

These approaches powered the TREC 2024 TOT track test queries and will continue into the 2025 track (trec-tot.github.io). Joyful collaboration with Yifan He @841io.bsky.social Jaime Arguello, and @bmitra.bsky.social ! #SIGIR #TREC #TOT

Did you know? Gestures used to express universal concepts—like wishing for luck—vary DRAMATICALLY across cultures? 🤞means luck in US but deeply offensive in Vietnam 🚨 📣 We introduce MC-SIGNS, a test bed to evaluate how LLMs/VLMs/T2I handle such nonverbal behavior! 📜: arxiv.org/abs/2502.17710

🎉Microsoft Research New England is hiring a predoctoral research assistant to work with @nancybaym.bsky.social, Tarleton Gillespie, and @marylgray.bsky.social on issues related to the dynamics of technology and society. 🎉 socialmediacollective.org/2025/01/22/s...

Join our team at Mila and help build trustworthy AI! We're hiring Postdocs & Interns to work on a cutting-edge project tackling misinformation. Combine LLMs, RAG, and temporal graphs to assess online credibility. PhD or strong CS/ML background needed. Apply here: forms.gle/ATdvNcZxYFcd...

🎉students studying AI and culture🎉 if you had an audience with senior folks from computer science, social science, and the humanities studying AI and culture, what questions would you ask them about the field, careers, skill sets, etc? students only! forms.gle/tZY5tJW2gqVd...

📢 We are hiring a postdoctoral fellow in ML and Environmental Data Science! 📈🌏 This interdisciplinary position is co-hosted by the CS department 💻 and the Environmental Data Science Innovation and Inclusion Lab (ESIIL) 🌱 at CU Boulder 🦬. Deadline coming up soon! jobs.colorado.edu/jobs/JobDeta...

I'm recruiting 1-2 PhD students to work with me at the University of Colorado Boulder! Looking for creative students with interests in #NLP and #CulturalAnalytics. Boulder is a lovely college town 30 minutes from Denver and 1 hour from Rocky Mountain National Park 😎 Apply by December 15th!

same but retrieval and #SIGIR at #NeurIPS and #NLProc conferences.

🎊 Excited to share the program for our Algorithmic Fairness through the lens of Metrics and Evaluation workshop at #NeurIPS2024! 📆 Join us Sat, Dec 14 for a day full of discussions on all things fairness and bias! We have a great program planned. To start, see our amazing list of speakers:👇

📆 AFME workshop: Sat, Dec 14 in room 111-112 Join our expert panellists* for a timely discussion on “Rethinking fairness in the era of large language models”!! * @jessicaschrouff.bsky.social, @sethlazar.org, Sanmi Koyejo, Hoda Heidari

Heading to #NeurIPS2024 to present our ‘Fair RAG’ paper at the #AFME2024 workshop! Let's talk about RAG, Information Retrieval, and Fairness. Honored that our paper was selected as one of the Top 5 Spotlight Papers! 🎉 Let’s connect and chat! Paper: arxiv.org/abs/2409.11598

Today we'll be presenting the Tutorial on Retrieval-Enhanced Machine Learning (REML). Come by to learn about the emerging design patterns in this space and see how to use retrieval beyond RAG. In collaboration w/ the amazing @841io.bsky.social @teknology.bsky.social Alireza Salemi and Hamed Zamani.