JHU computer scientists including @williamjurayj.bsky.social propose a method that allows #AI models to spend more time thinking through problems & uses a confidence score to determine when the AI should say "I don't know" rather than risking a wrong answer, which is crucial for high-stakes domains.

You can't just be right, you have to know you're right. Good advice for LLMs, according to new Johns Hopkins research. Sometimes no answer is better than a wrong one - life or death choices in medicine, for example, or big financial decisions. 🧵

🚨 You are only evaluating a slice of your test-time scaling model's performance! 🚨 📈 We consider how models’ confidence in their answers changes as test-time compute increases. Reasoning longer helps models answer more confidently! 📝: arxiv.org/abs/2502.13962

I’d say a key factor is whether a person’s put in a good faith effort to be right for the right reasons. But I’m to other explanations!

I noticed a lot of starter packs skewed towards faculty/industry, so I made one of just NLP & ML students: go.bsky.app/vju2ux Students do different research, go on the job market, and recruit other students. Ping me and I'll add you!