I was struck with an incredible thought: The Subword Tolkienizer.

I was struck with an incredible thought: The Subword Tolkienizer.

🎉 Congratulations to all #EMNLP2025 award winners 🎉 Starting with the ✨Best Paper award ✨: "Infini-gram mini: Exact n-gram Search at the Internet Scale with FM-Index" by Hao Xu, Jiacheng Liu, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi aclanthology.org/2025.emnlp-m... 1/n

Got to the part of "temperature" and I'm aware that a higher temperature == less predictable but never knew why. Turns out it's very simple. Before the "score" for a set of tokens is turned into a probability distribution it's divided by the temperature. Higher values "flatten" the distribution.

Just added my book, "Theory of Computing: An Open Introduction" to OER Commons, and working on getting it listed in Canadian repositories too. One step closer to making education more open and accessible to everyone! oercommons.org/courses/theo...

It’s grad school application season, and I wanted to give some public advice. Caveats: -*-*-*-* 
> These are my opinions, based on my experiences, they are not secret tricks or guarantees 
> They are general guidelines, not meant to cover a host of idiosyncrasies and special cases

This is very high on my list of advice for PhD applicants. I've written two SoP (masters and PhD) and the similarities between the things I wrote about in the SoP and the things I wrote my theses on ends roughly at "written in English".

Mistake 3, cont': people worry they narrow down by proposing specific questions ("What if this is not the EXACT thing I want to work on in grad school?"). But an SoP is not a *contract*, it will not be waved in front of you when starting grad school.

y'all seem to really like baseball bsky.social/about/blog/1...

Presenting today our work "Unsupervised Word-level Quality Estimation Through the Lens of Annotator (Dis)agreement" at the Machine Translation morning session (Room A301, 11:45 China time). See you there! 🤗 Paper: aclanthology.org/2025.emnlp-m... Slides/video/poster: underline.io/events/502/s...

why intern at Ai2? 🐟interns own major parts of our model development, sometimes even leading whole projects 🐡we're committed to open science & actively help our interns publish their work reach out if u wanna build open language models together 🤝 links 👇

For more about things like this, here's an article (actually it's a series) that goes in-depth into this topic for a Canadian election. One of my favorite articles to share.

The For You feed is nice but it's really sensitive (?). Like every now and then my feed just explodes with some niche topic that's seemingly unrelated to anything I've interacted with.

One of the hardest Pytorch bug I had to debug is due to how the logsumexp behave with -inf masked inputs. Consider the following example. I build a vector of 3 logits, and each logit is the result of a logsumexp.

I wrote a short blog post about masked softmax layers in PyTorch (i.e., when you have structural constraints that tell you some classes _must_ have probability zero). This was based on a real bug I found in a neural chess model implementation.

NLP evaluation is often detached from practical applications. Today I extrinsically evaluated one WMT25 translation system on the task of getting hair done without knowing Chinese. Yes you got 67 BLEU points but is the resulting hair slaying? 💇 See the result on one datapoint (my head) at EMNLP.

Let's talk about eval (automatic or human) and multilinguality at #EMNLP in Suzhou! 🇨🇳 - Efficient evaluation (Nov 5, 16:30, poster session 3) - MT difficulty (Nov 7, 12:30, findings 3) - COMET-poly (Nov 8, 11:00, WMT) (DM to meet 🌿 )

Catch me way under par in the Code Golf Masters after this change arrives.

I wrote a short blog post about masked softmax layers in PyTorch (i.e., when you have structural constraints that tell you some classes _must_ have probability zero). This was based on a real bug I found in a neural chess model implementation.

Need to establish a norm against making the manifold chip-coloured so im not hungry reading papers

Wow, Saddam Hussein had the same interior decorator as an Airbnb I went to in Jeju once. *Excuse the awkward angle, it's a screenshot from a video.

The Llama2 tokenizer is certainly not helping with this problem.

Me, when I see a building.

The project that started my whitespace obsession... #EMNLP2025 While we've all been worrying about tokenizers, lurking in the background has been the preprocessing *before* tokenization. Poems break standard HTML-to-text linearization systems, and we find that multimodal models aren't a solution.

theoriticians with side hustles

theoriticians with side hustles

📢 Join 1M+ learners worldwide in the 5-Day AI Agents Intensive with Google! Learn from Google experts through theory sessions, hands-on labs, a capstone project, and connect with a vibrant community - all at no-cost! ⏳ Don’t miss out: rsvp.withgoogle.com/events/googl...

Our #NeurIPS2025 paper shows that even comparable monolingual tokenizers have different compression rates across languages. But by getting rid of whitespace tokenization and using a custom vocab size for each language, we can reduce token premiums. Preprint out now!

Someone is training neural nets on their new Pebble 2 Duo, of course... github.com/breitburg/pe...

He was a real one, no doubt. > To protest against the Vietnam War, he gave lectures on category theory in the forests surrounding Hanoi while the city was being bombed.

I'm trying to get one of these from the Tōyoko line's Shin-Maruko station (新丸子駅), since it has the same spelling as my name (but I use マルコ, ofc).

He was a real one, no doubt. > To protest against the Vietnam War, he gave lectures on category theory in the forests surrounding Hanoi while the city was being bombed.

TIL Grothendieck wrote a 10 page essay on kimchi. #mathsky

"Look, I know this looks like abstract hobby math, but if we ever figure out how to make sand think, this is going to be really important for compressing cat videos" - some guy in 1756 probably

I busted out my old thinkpad today to find a project I built a long time ago. I love my macbook, but the thinkpad just hits different.

Can you predict when Lou Bega's greatest hit, Mambo No. 5, was released based on the names of all the women mentioned in the song? Weirdly yes. yawpr.substack.com/p/project-a-... #databs #rstats

Z.ai released a paper very similar to DeepSeek-OCR on the same exact day (a few hours earlier afaict) Glyph is just a framework, not a model, but they got Qwen3-8B (128k context) to handle over 1 million context by rendering input as images arxiv.org/abs/2510.17800

Curious what you can do with Typst's HTML export right now? How about write a blog? This article tells you how: blog.sillyfreak.space/2025/06/29/r...

JuliaHealth is on @hf.co! 🤗 If you are interested in #julialang, #llm or #agentic workflows, and how #GenAI can be used within public health, medical informatics, and survey-based research, drop us a line or a follow! 🤓 How are you using GenAI in your medical research? #opensource #medsky #data

📢 Announcing the First Workshop on Multilingual and Multicultural Evaluation (MME) at #EACL2026 🇲🇦 MME focuses on resources, metrics & methodologies for evaluating multilingual systems! multilingual-multicultural-evaluation.github.io 📅 Workshop Mar 24–29, 2026 🗓️ Submit by Dec 19, 2025

Slides from my #PyBay2025 talk: "Just because AI can write your tests...should it?" pamelafox.github.io/my-py-talks/...

The New York Times recently introduced daily puzzles called Pips. You place the dominoes on the grid so the numbers satisfy the labels. I solved Pips with cool software called a constraint solver. You give the constraints, e.g. "sum to 8", and it "magically" finds a solution. Let's look closer...

I'm on the way to SF for @pybay.bsky.social via CalTrain. So far a much nicer experience than the last time I took CalTrain (but not yet Japan level). Will post about some of the interesting talks. #PyBay

Welcome new bsky users! Here are some starter packs/feeds for ppl interested in Theoretical Computer Science & Math: MathSky feed: bsky.app/profile/did:... TCS: go.bsky.app/2jS2SGw (by me) Math: go.bsky.app/66Us7WW by @tienchihmath.bsky.social go.bsky.app/GiRFQ8D by @visualalgebra.bsky.social

𝑵𝒆𝒘 𝒃𝒍𝒐𝒈𝒑𝒐𝒔𝒕! A rundown of some cool papers I got to chat about at #COLM2025 and some scattered thoughts saxon.me/blog/2025/co...

🍞+😬

Here is a story about a AAAI paper that got 5 human reviews and 1 AI review. The AI generated review contained a page-long counterexample to a proof in the paper. The AI’s counter example contained hallucinated errors. How does one rebut an AI reviewer? www.linkedin.com/posts/omerbp...

This is why I always wear a hat: to measure the magnitude of my thoughts.

Pokemon is truly the pareto frontier of agent research - The RPG requires an autonomous embodied agentic agent with perception, planning, memory, and control - VGC and Gen 9 OU penalize erroneous actions with fast-paced opponent-modeling in short games (1/3)