Dirk Hovy
@dirkhovy.bsky.social
๐ค 614
๐ฅ 327
๐ 48
Professor
@milanlp.bsky.social
for
#NLProc
, compsocsci,
#ML
Also at
http://dirkhovy.com/
pinned post!
๐จ(Software) Update: In my PhD, I had a side project to fix an annoying problem: when you ask 5 people to label the same thing, you often get different answers. But in ML (and lots of other analyses), you still need a single aggregated answer. Using the majority vote is easyโbut often wrong. 1/N
loading . . .
GitHub - dirkhovy/MACE: Multi-Annotator Competence Estimation tool
Multi-Annotator Competence Estimation tool. Contribute to dirkhovy/MACE development by creating an account on GitHub.
https://github.com/dirkhovy/MACE
about 2 months ago
6
75
14
reposted by
Dirk Hovy
MilaNLP Lab
6 days ago
#MemoryModay
#NLProc
'Comparing Bayesian Models of Annotation' by Paun et al. dives into corpus annotation, evaluating six models' predictiveness and accuracy. Essential for navigating annotators and item difficulties.
loading . . .
Comparing Bayesian Models of Annotation
Silviu Paun, Bob Carpenter, Jon Chamberlain, Dirk Hovy, Udo Kruschwitz, Massimo Poesio. Transactions of the Association for Computational Linguistics, Volume 6. 2018.
https://aclanthology.org/Q18-1040
0
8
2
reposted by
Dirk Hovy
MilaNLP Lab
5 days ago
๐ข Call for Abstracts! Towards a Safer Web for Women (co-located with
#WebSci26
) ๐ Braunschweig ๐ฉ๐ช | 26 May 2026 Theme: Preventive approaches to womenโs online safety ๐ Deadline: 27 March 2026 ๐
forms.gle/tYheEgSwGecf...
๐
tsww26.github.io
0
5
4
reposted by
Dirk Hovy
MilaNLP Lab
3 days ago
#TBT
#NLProc
'Predicting News Headline Popularity' by Lamprinidis, Hardt,
@dirkhovy.bsky.social
(2018) shows neural networks perform similar to Logistic Regression in prediction.
loading . . .
Predicting News Headline Popularity with Syntactic and Semantic Knowledge Using Multi-Task Learning
Sotiris Lamprinidis, Daniel Hardt, Dirk Hovy. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018.
https://aclanthology.org/D18-1068
0
2
2
One of my favorite studies of the last few years! Great read (albeit with a side of worrying implications for surveys)
add a skeleton here at some point
3 days ago
0
5
2
One of my favorite interdisciplinary projects (with
@questoph.bsky.social
). Plus: colorful maps!
add a skeleton here at some point
9 days ago
0
3
1
reposted by
Dirk Hovy
MilaNLP Lab
10 days ago
#TBT
#NLProc
'Capturing Regional Variation with Distributed Place Representations and Geographic Retrofitting' by
@dirkhovy.bsky.social
and Christoph Purschke (2018) highlights how social class and background impact technology performance.
#TechInclusion
loading . . .
Capturing Regional Variation with Distributed Place Representations and Geographic Retrofitting
Dirk Hovy, Christoph Purschke. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018.
https://aclanthology.org/D18-1469
0
3
4
reposted by
Dirk Hovy
Tiancheng Hu
13 days ago
4/7 We argue these aren't separate bugs. They're four facets of the same problem: ๐ด Probabilistic โ can't match requested distributions ๐ Semantic โ confidence โ correctness ๐ต Distributional โ output diversity collapse ๐ข Metacognitive โ can't assess its own competence
1
2
1
reposted by
Dirk Hovy
Tiancheng Hu
13 days ago
1/7 ๐งต The GPT-4 technical report featured detailed calibration curves. Since then, not a single major model release has reported calibration. The field quietly stopped measuring whether models know what they don't know. Our new position paper argues this is a mistake. Here's why.
1
8
2
reposted by
Dirk Hovy
MilaNLP Lab
19 days ago
We were thrilled to host
@mtutek.bsky.social
at our lab last week. His talk "From Internals to Integrity: How Insights into Transformer LMs Improve Safety, Interpretability, and Explanation Faithfulness" led to great discussions! ๐
#Transformers
#AISafety
#ExplainableAI
#MLResearch
#NLProc
0
18
3
reposted by
Dirk Hovy
EACL 2026
19 days ago
Call for Virtual Registration Subsidies for
#EACL26
๐ โ ๏ธ Not for paper registrants ๐ Apply by Feb 27, 2026 (AoE) ๐ฉ Decisions by Mar 2, 2026
2026.eacl.org/calls/virtua...
Donโt register before hearing back if you apply!
loading . . .
Call for Virtual Registration Subsidies
Official website for the 2026 Conference of the European Chapter of the Association for Computational Linguistics
https://2026.eacl.org/calls/virtual-subsidies/
1
6
5
reposted by
Dirk Hovy
UKP Lab
25 days ago
๐๐งฉ ๐๐ฒ๐๐ผ๐ป๐ฑ ๐๐ฒ๐ป๐ฐ๐ต๐บ๐ฎ๐ฟ๐ธ๐: ๐๐ผ๐ ๐๐ผ ๐๐๐ฎ๐น๐๐ฎ๐๐ฒ ๐ ๐ฒ๐ป๐๐ฎ๐น ๐๐ฒ๐ฎ๐น๐๐ต ๐๐ ๐ฅ๐ฒ๐๐ฝ๐ผ๐ป๐๐ถ๐ฏ๐น๐ AI for mental health is a high-stakes area: its evaluation needs to meet the highest expectations. The new preprint ๐๐ฆ๐ด๐ฑ๐ฐ๐ฏ๐ด๐ช๐ฃ๐ญ๐ฆ ๐๐ท๐ข๐ญ๐ถ๐ข๐ต๐ช๐ฐ๐ฏ ๐ฐ๐ง ๐๐ ๐ง๐ฐ๐ณ ๐๐ฆ๐ฏ๐ต๐ข๐ญ ๐๐ฆ๐ข๐ญ๐ต๐ฉ, written by an interdisciplinary team spanning AI [...]
1
3
3
reposted by
Dirk Hovy
Debora Nozza
27 days ago
Honored to give my first keynote at
#IRCDL2026
on February 19th. Iโll talk about how LLMs have shifted from productivity tools to everyday sources of info & personal guidance and what that means for risk, trust, bias, and alignment.
ircdl2026.unimore.it
0
14
2
reposted by
Dirk Hovy
Cambridge University Press Political Science & IR
27 days ago
#OpenAccess from
@politicsgenderj.bsky.social
- Male Agency? Analyzing Fatherhood Roles in Swedish Parliamentary Documents, 1993โ2021 - https://cup.org/40el36q - Lena Wรคngnerud, Elin Naurin,
@dirkhovy.bsky.social
#OpenAccess
Lorenzo Lupo & Oscar Magnusson
#FirstView
0
6
2
reposted by
Dirk Hovy
MilaNLP Lab
about 2 months ago
#MemoryModay
#NLProc
@gattanasio.cc
et al. study asks 'Is It Worth the (Environmental) Cost?' analyzing continuous training for language models. Balances benefits, environmental impacts, for responsible use.
#AI
#Sustainability
arxiv.org/pdf/2210.07365
loading . . .
https://arxiv.org/pdf/2210.07365
0
7
4
reposted by
Dirk Hovy
รtienne Ollion
about 2 months ago
What are the main issues discussed in a set of documents? Weโve just released a step-by-step BERTopic tutorial. We also launch a new page, gathering various NLP tutorials for social scientists. ๐
www.css.cnrs.fr/tutorials-an...
loading . . .
Tutorials and Resources โ CSS @ IP-Paris
Site web de l'axe sciences sociales computationnelles du CREST-CNRS. Cours et tutoriels pour l'analyse des donnรฉes numรฉriques en sciences sociales.
https://www.css.cnrs.fr/tutorials-and-resources/
3
48
25
reposted by
Dirk Hovy
David Mimno
about 2 months ago
Citation is the foundation of academic promotion. Itโs noisy, sure, but its integrity is worth fighting for. Hallucinated citations should be a desk reject.
add a skeleton here at some point
1
27
5
reposted by
Dirk Hovy
David Jurgens
about 2 months ago
The second new class I'm teaching is a very experimental graduate level seminar in CSE: "Building Small Language Models". I taught the grad level NLP class last semester (so fun!) but students wanted moreโwhich of these new ideas work, and which work for SLMs?
jurgens.people.si.umich.edu/CSE598-004/
loading . . .
CSE 598-004 - Building Small Language Models
http://jurgens.people.si.umich.edu/CSE598-004/
2
32
10
reposted by
Dirk Hovy
MilaNLP Lab
about 2 months ago
๐ MilaNLP 2025 Wrapped ๐ Lots of learning, building , sharing, and growing together ๐ฑ
#NLProc
0
10
4
reposted by
Dirk Hovy
MilaNLP Lab
about 2 months ago
โณ Deadline approaching! Weโre hiring 2 fully funded postdocs in
#NLP
. Join the MilaNLP team and contribute to our upcoming research projects (SALMON & TOLD) ๐ Details + how to apply:
milanlproc.github.io/open_positio...
โฐ Deadline: Jan 31, 2026
0
11
11
๐จ(Software) Update: In my PhD, I had a side project to fix an annoying problem: when you ask 5 people to label the same thing, you often get different answers. But in ML (and lots of other analyses), you still need a single aggregated answer. Using the majority vote is easyโbut often wrong. 1/N
loading . . .
GitHub - dirkhovy/MACE: Multi-Annotator Competence Estimation tool
Multi-Annotator Competence Estimation tool. Contribute to dirkhovy/MACE development by creating an account on GitHub.
https://github.com/dirkhovy/MACE
about 2 months ago
6
75
14
New year, new job? If that is your current mantra, check the open postdoc positions with Debora Nozza and me at our lab. Deadline is January 31st.
milanlproc.github.io/open_positio...
loading . . .
Postdoctoral Researcher โ NLP (2 positions) | MilaNLP Lab @ Bocconi University
Two Postdoctoral Researcher positions โ Deadline January 31st, 2026
https://milanlproc.github.io/open_positions/postdoc_tef/
about 2 months ago
0
11
11
reposted by
Dirk Hovy
MilaNLP Lab
3 months ago
๐ Weโre opening 2 fully funded postdoc positions in
#NLP
! Join the MilaNLP team and contribute to our upcoming research projects. ๐ More details:
milanlproc.github.io/open_positio...
โฐ Deadline: Jan 31, 2026
0
19
15
Happy to have contributed to this
add a skeleton here at some point
3 months ago
0
3
0
reposted by
Dirk Hovy
MilaNLP Lab
3 months ago
#MemoryModay
#NLProc
Countering Hateful and Offensive Speech Online - Open Challenges" by Plaza-Del-Arco, @debora_nozza, Guerini, Sorensen, Zampieri, 2024 is a tutorial on the challenges and solutions for detecting and mitigating hate speech.
loading . . .
Countering Hateful and Offensive Speech Online - Open Challenges
Flor Miriam Plaza-del-Arco, Debora Nozza, Marco Guerini, Jeffrey Sorensen, Marcos Zampieri. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Tutorial Abstracts.โฆ
https://aclanthology.org/2024.emnlp-tutorials.2/
0
4
2
reposted by
Dirk Hovy
MilaNLP Lab
3 months ago
#MemoryModay
#NLProc
Uma, A. N. et al. examine AI model training in 'Learning from Disagreement: A Survey'. Disagreement-handling methods' performance is shaped by evaluation methods & dataset traits.
loading . . .
https://jair.org/index.php/jair/article/view/12752
0
4
2
reposted by
Dirk Hovy
MilaNLP Lab
3 months ago
#TBT
#NLProc
#MachineLearning
#SafetyFirst
'Safety-Tuned LLaMAs: Improving LLMs Safety' by Bianchi et al. explores training LLMs for safe refusals, warns of over-tuning.
loading . . .
Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large...
Training large language models to follow instructions makes them perform better on a wide range of tasks and generally become more helpful. However, a perfectly helpful model will follow even the most...
https://arxiv.org/abs/2309.07875
0
4
2
Come work with
@deboranozza.bsky.social
, me, and the lab in Milan!
add a skeleton here at some point
3 months ago
0
6
3
reposted by
Dirk Hovy
Women in AI Research - WiAIR
3 months ago
We don't actually trust AI. We trust the companies behind it. As Maria Antoniak notes, every "private" chat flows through corporate systems with long histories of data misuse. If we care about AI ethics, we need to name power, not anthropomorphize models.
loading . . .
1
54
18
reposted by
Dirk Hovy
jake hofman
3 months ago
We're hiring interns in the Computational Social Science group at Microsoft Research NYC! If you're interested in designing AIโbased systems and understanding their impact at both individual and societal scales, apply here by Jan 9, 2026:
apply.careers.microsoft.com/careers/job/...
loading . . .
Research Intern - Computational Social Science | Microsoft Careers
Research Interns put inquiry and theory into practice. Alongside fellow doctoral candidates and some of the world's best researchers, Research Interns learn, collaborate, and network for life. Researc...
https://apply.careers.microsoft.com/careers/job/1970393556639564
0
21
18
After I shared โHow to professorโ last year, some people asked for a similar post on writing. Now I finally got around to typing up our lab's writing workshop slides. It covers basic advice for research papers and grant applications. Curious? Read it here:
dirkhovy.com/post/2025_11...
loading . . .
How to Write Gooder | Dirk Hovy
After publishing โ How to professorโ, several people said they found it helpful, and asked whether I had a similar post on writing. Luckily, we have held an annual writing workshop in the lab for the last few years, so there already was a presentation.
https://dirkhovy.com/post/2025_11_25/
3 months ago
1
12
3
reposted by
Dirk Hovy
MilaNLP Lab
3 months ago
#TBT
#NLProc
'Respectful or Toxic?' by Plaza-del-Arco, @debora &
@dirkhovy.bsky.social
(2023) explores zero-shot learning for multilingual hate speech detection. Highlights prompt & model choice for accuracy.
#AI
#LanguageModels
#HateSpeechDetection
loading . . .
Respectful or Toxic? Using Zero-Shot Learning with Language Models to Detect Hate Speech
Flor Miriam Plaza-del-arco, Debora Nozza, Dirk Hovy. The 7th Workshop on Online Abuse and Harms (WOAH). 2023.
https://aclanthology.org/2023.woah-1.6
0
2
2
reposted by
Dirk Hovy
MilaNLP Lab
3 months ago
#MemoryModay
#NLProc
'Leveraging Social Interactions to Detect Misinformation on Social Media' by Fornaciari et al. (2023) uses combined text and network analysis to spot unreliable threads.
loading . . .
https://arxiv.org/pdf/2304.02983
0
3
2
reposted by
Dirk Hovy
Manoel Horta Ribeiro
3 months ago
The Center for Information Technology Policy at Princeton invites applications for a Postdoctoral Fellow to work with Andy Guess (Politics/SPIA), Brandon Stewart (Sociology), and me (CS).
puwebp.princeton.edu/AcadHire/app...
Please apply before Sunday, the 13th of December!
0
16
10
reposted by
Dirk Hovy
MilaNLP Lab
3 months ago
#MemoryModay
#NLProc
'Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models' by
@paul-rottger.bsky.social
et al. (2022). A suite of tests for 10 languages.
loading . . .
Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models
Paul Rรถttger, Haitham Seelawi, Debora Nozza, Zeerak Talat, Bertie Vidgen. Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH). 2022.
https://aclanthology.org/2022.woah-1.15
0
3
2
reposted by
Dirk Hovy
MilaNLP Lab
3 months ago
#TBT
#NLProc
'Compromesso! Italian Many-Shot Jailbreaks Undermine LLM Safety' by Pernisi,
@dirkhovy.bsky.social
,
@paul-rottger.bsky.social
(2024). Paper highlights LLM vulnerability through Italian demos, more demos = more attack chances.
loading . . .
Compromesso! Italian Many-Shot Jailbreaks undermine the safety of Large Language Models
Fabio Pernisi, Dirk Hovy, Paul Rรถttger. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop). 2024.
https://aclanthology.org/2024.acl-srw.29/
0
3
2
reposted by
Dirk Hovy
Greg Durrett
3 months ago
๐ข Postdoc position ๐ข Iโm recruiting a postdoc for my lab at NYU! Topics include LM reasoning, creativity, limitations of scaling, AI for science, & more! Apply by Feb 1. (Different from NYU Faculty Fellows, which are also great but less connected to my lab.) Link in ๐งต
2
21
13
reposted by
Dirk Hovy
Tanise Ceron
3 months ago
I will be
@euripsconf.bsky.social
this week to present our paper as non-archival at the PAIG workshop (Beyong Regulation: Private Governance & Oversight Mechanisms for AI). Very much looking forward to the discussions! If you are at
#EurIPS
and want to chat about LLM's training data. Reach out!
add a skeleton here at some point
0
9
4
reposted by
Dirk Hovy
MilaNLP Lab
4 months ago
Another exhausting day in the labโฆ conducting very rigorous panettone analysis. Pandoro was evaluated too, because we believe in fair experimental design.
0
23
7
reposted by
Dirk Hovy
MilaNLP Lab
4 months ago
#TBT
#NLProc
'@donyarn.bsky.social &
@dirkhovy.bsky.social
's 2024 paper, 'Conversations as a Source for Teaching Scientific Concepts' turns video dialogues into effective teaching tools.'
loading . . .
https://arxiv.org/pdf/2404.10475
0
3
2
reposted by
Dirk Hovy
EACL 2026
4 months ago
๐ขSubmissions are open for EACL 2026 SRW **pre-submission mentorship**! Get valuable feedback on your research paper draft from experienced mentors before the deadline. Don't miss this chance to refine your work! ๐ Submit here:
openreview.net/group?id=eac...
@eaclmeeting
#NLProc
loading . . .
EACL 2026 SRW Mentorship
Welcome to the OpenReview homepage for EACL 2026 SRW Mentorship
https://openreview.net/group?id=eacl.org/EACL/2026/SRW_Mentorship
0
3
1
reposted by
Dirk Hovy
brendan oโconnor
4 months ago
Here at UMass Amherst CICS, weโre searching for TT faculty in NLP โ see the link from
www.cics.umass.edu/about/employ...
Iโm happy to answer questions of course, too!
loading . . .
Faculty Positions
Open tenure-track and teaching faculty positions in computer science and informatics at the Manning College of Information and Computer Sciences
https://www.cics.umass.edu/about/employment/faculty-positions
2
18
14
reposted by
Dirk Hovy
MilaNLP Lab
4 months ago
#MemoryModay
#NLProc
'Hey Siri. Ok Google. Alexa: A topic modeling of user reviews for smart speakers,' by Nguyen &
@dirkhovy.bsky.social
decodes speaker reviews for user preferences using topic models. Domain knowledge needed for market analysis.
loading . . .
Hey Siri. Ok Google. Alexa: A topic modeling of user reviews for smart speakers
Hanh Nguyen, Dirk Hovy. Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019). 2019.
https://aclanthology.org/D19-5510
0
3
2
reposted by
Dirk Hovy
MilaNLP Lab
4 months ago
โTeacher Demonstrations in a BabyLMโs Zone of Proximal Development for Contingent Multi-Turn Interactionโ selected for an Outstanding Paper Award at the BabyLM Challenge & Workshop!
0
7
3
reposted by
Dirk Hovy
MilaNLP Lab
4 months ago
What an inspiring week at
#EMNLP2025
in Suzhou๐จ๐ณ! Huge thanks to the organizers and everyone who stopped by our poster/talk!
1
18
5
reposted by
Dirk Hovy
Dallas Card
4 months ago
See also
@manoelhortaribeiro.bsky.social
's post on this same topic:
doomscrollingbabel.manoel.xyz/p/labeling-d...
loading . . .
Labeling Data with Language Models: Trick or Treat?
Large language models are now labeling data for us.
https://doomscrollingbabel.manoel.xyz/p/labeling-data-with-language-models
0
2
1
reposted by
Dirk Hovy
Dallas Card
4 months ago
Trying an experiment in good old-fashioned blogging about papers:
dallascard.github.io/granular-mat...
loading . . .
Language Model Hacking - Granular Material
https://dallascard.github.io/granular-material/post/language-model-hacking/
3
29
9
reposted by
Dirk Hovy
MilaNLP Lab
4 months ago
#TBT
#NLProc
' Attanasio et al. study asks 'Is It Worth the (Environmental) Cost?' analyzing continuous training for language models. Balances benefits, environmental impacts, for responsible use.
#Sustainability
'
loading . . .
https://arxiv.org/pdf/2210.07365
0
3
3
reposted by
Dirk Hovy
MilaNLP Lab
4 months ago
#MemoryModay
#NLProc
' 'State of Profanity Obfuscation in NLP Scientific Publications' probes bias in non-English papers.
@deboranozza.bsky.social
&
@dirkhovy.bsky.social
(2023) propose 'PrOf' to aid authors & improve access.
loading . . .
The State of Profanity Obfuscation in Natural Language Processing Scientific Publications
Debora Nozza, Dirk Hovy. Findings of the Association for Computational Linguistics: ACL 2023. 2023.
https://aclanthology.org/2023.findings-acl.240
0
4
2
reposted by
Dirk Hovy
MilaNLP Lab
5 months ago
#TBT
#NLProc
Explore 'Wisdom of Instruction-Tuned LLM Crowds' by Plaza et al. LLM labels outperform single models in tasks & languages. But few-shot can't top zero-shot. Supervised models rule.
loading . . .
Wisdom of Instruction-Tuned Language Model Crowds. Exploring Model Label Variation
Flor Miriam Plaza-del-Arco, Debora Nozza, Dirk Hovy. Proceedings of the 3rd Workshop on Perspectivist Approaches to NLP (NLPerspectives) @ LREC-COLING 2024. 2024.
https://aclanthology.org/2024.nlperspectives-1.2
0
2
2
reposted by
Dirk Hovy
MilaNLP Lab
4 months ago
#MemoryModay
#NLProc
'Universal Joy: A Data Set and Results for Classifying Emotions Across Languages' by Lamprinidis et al. (2021) explores how AI research affects our planet.
loading . . .
Universal Joy A Data Set and Results for Classifying Emotions Across Languages
Sotiris Lamprinidis, Federico Bianchi, Daniel Hardt, Dirk Hovy. Proceedings of the Eleventh Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. 2021.
https://aclanthology.org/2021.wassa-1.7
0
6
2
Load more
feeds!
log in
