Coding agents are real users of the @hf.co Hub! They're searching for models, building and pushing datasets, training models on Jobs, spinning up Spaces... Now there's public data: each agent's share of Hub traffic, updated monthly 👇

The latest @commoncrawl.bsky.social crawl indexes way more than HTML pages. 20.9M PDFs. Plus calendars, BibTeX, markdown... The whole index now lives in a @hf.co Bucket, so I pulled this with one SQL query straight over it (new S3 API). 2.1B rows, nothing downloaded, $0 to read.

You can now use 100s of tools with @hf.co Buckets, thanks to the new S3 API! Usually just one or two lines to change. huggingface.co/docs/hub/sto...

Fairly benchmarking OCR models is hard! Ran a few newer OCR models on @ai2.bsky.social's olmOCR-bench "old scans" subset. Worth knowing: the score "punishes" models for extracting too much (letterheads, stamps, etc.)

If libraries, archives and museums pooled their (labelled) data, they could build state-of-the-art open models for the things they actually care about! I tried a small version: one open model (NuExtract-3, 4B) fine-tuned to read archival index cards across several collections.

Excited to share this. @neel2112.bsky.social, @mariaa.bsky.social, and I analyzed 500K anonymous ChatGPT convos (shared w/ consent from WildChat) to see if people were generating fiction. We found tons of stories, fanfiction & erotica. Many users iterated on the same stories for days and weeks.

Say, for example, if we had 3 examples of labeled city directory data from every state in the US, we could easily train a generic city directory parser! (I’m actually working on this, and I’ll have it done relatively soon, but @danielvanstrien.bsky.social’s point holds in so many other cases!)

If libraries, archives and museums pooled their (labelled) data, they could build state-of-the-art open models for the things they actually care about! I tried a small version: one open model (NuExtract-3, 4B) fine-tuned to read archival index cards across several collections.

Ran the same 650M model across 7 languages: German, Polish, Finnish, Russian, Serbian, Estonian, Swedish via @europeana.bsky.social newspapers. Still some challenging layouts, but pretty mind-blowing to me that a 650M model is doing this when a year ago 72B VLMs failed very often.

I think VLM-based OCR might finally be close to working on historic newspapers! Many models I've tried before failed i.e. hallucinations, repetition loops, context overflow. Surya OCR 2 (a 650M model!) does a very good job!

I've been offline more than usual because of paper and grant deadlines plus work travel, but I'm back with my first real newsletter today! Click for rambling about infrastructure, benchmarks, Babylonian tablets, and more.

Can the new DiffusionGemma model help fix broken OCR? In theory, denoising tokens in parallel could work better for OCR correction since context is seen upfront? Pointed it at 19th-century newspaper OCR. It corrected better than the autoregressive baseline — at ~8x the speed.

Can the new DiffusionGemma model help fix broken OCR? In theory, denoising tokens in parallel could work better for OCR correction since context is seen upfront? Pointed it at 19th-century newspaper OCR. It corrected better than the autoregressive baseline — at ~8x the speed.

Got a digitised collection that needs OCR? uv-scripts is a set of single-file Python scripts that OCR a whole image dataset to markdown in one command — 20+ open VLMs to pick from, nothing to install but uv. github.com/davanstrien/...

Got a digitised collection that needs OCR? uv-scripts is a set of single-file Python scripts that OCR a whole image dataset to markdown in one command — 20+ open VLMs to pick from, nothing to install but uv. github.com/davanstrien/...

What could a rich ecosystem of small GLAM AI models enable? IMO: cheaper, better-fitted, more robust models. Example: I extended an existing @natlibscot.bsky.social archival card detector to 4 collections to make a more generic index card detector. Took an hour or two and minimal $

Derived datasets are bigger on Hugging Face Hub than people realise. ~73% of analysed datasets on the Hub are derivatives of something else, i.e. cleaned, translated, extended, etc. Built an explorer that infers the missing lineage from content: huggingface.co/spaces/davan...

Built a useful GLAM model this week: flags blank index cards before expensive OCR. Cheap, scales, adapts. IMO the technical barrier is now very low, the main one is knowing how. So I wrote this up as a chapter: danielvanstrien.xyz/ai-patterns-for-glam/patterns/index-card-classifier.html

You can now run SQL over 2.19 BILLION web pages — zero download. @commoncrawl.bsky.social April 2026 crawl + URL index are on Hugging Face Storage Buckets. DuckDB reads it straight over hf:// — I counted all 2.19B in ~35s. Or point your own agent at it 👇 huggingface.co/spaces/davan...

The 'why' behind yesterday's demo: libraries don't (just) want searchable text from catalogue cards. They want structured records they can ingest into existing systems. Short blog + a live demo for this library use case: danielvanstrien.xyz/posts/2026/s...

NuExtract3 (4B, Apache-2.0) does OCR *and* structured extraction. Point it at a dataset of scanned index cards + a JSON schema → clean catalog JSON One command on huggingface Jobs. (or skip the schema for plain Markdown OCR) Script + dataset 👇

What is "AI for libraries" beyond a catalogue chatbot? IMO: design patterns (OCR, extraction, classification, search), and agents that both run them and develop the small models behind them. As part of work with @natlibscot.bsky.social started a book on this: danielvanstrien.xyz/ai-patterns-...

Nouvelle leçon ! 💫 Une nouvelle traduction de la leçon de @danielvanstrien.bsky.social, Kaspar Beelen, @melvinwevers.bsky.social, @thomassmits.bsky.social, et @kmcdono.bsky.social. doi.org/10.46430/phfr0041

Can an open-weight coding agent match Claude Code at training a domain-specific model? Same prompt. ~13 min each. Pushed to Hugging Face. Pi + Kimi K2.6 vs Claude Code + Opus 4.7. Task: classify NC session laws (1866-1967) as Jim Crow or not. Full write-up: danielvanstrien.xyz/posts/2026/a...

Can an open-weight coding agent match Claude Code at training a domain-specific model? Same prompt. ~13 min each. Pushed to Hugging Face. Pi + Kimi K2.6 vs Claude Code + Opus 4.7. Task: classify NC session laws (1866-1967) as Jim Crow or not. Full write-up: danielvanstrien.xyz/posts/2026/a...

2 million book titles visualised as an interactive Embedding Atlas. Fiction, History, Science — each forms its own cluster. The trick: mount the same @hf.co Bucket to a GPU Job AND a Space. Job writes, Space reads. No uploads!

Segment any object in an image dataset with a text prompt — one command. uv run segment-objects.py data output --class-name deer Pixel-level masks via SAM3. Perfect for agents building their own training data. Runs on @hf.co Jobs. huggingface.co/datasets/uv-...

Wanted to share a personal project I've been working on: I love classic literature, and I'm also taking an AI class here at UIUC, so I trained an LLM from scratch on Victorian lit from 1837 to 1899. This is Mr. Chatterbox, the Victorian Gentleman Chatbot: huggingface.co/spaces/tvent...

You never know what data will be used for! I uploaded a @britishlibrary.bsky.social dataset to Hugging Face in 2022. IIRC one of my first PR to a HF repo! 4 years later, someone trains a Victorian chatbot on it More libraries should be sharing their public domain collections for AI to build on!

Is olmOCR-bench getting close to saturation? Top score is now 85.9%. Yesterday, Datalab took #1 with chandra-ocr-2. A year ago, the best was 79. Visualised the race to get there using @hf.co leaderboard data

One of the nicest things about Nvidia model releases is that they ship the training data. What does it look like? I sampled 250k examples from 24 datasets in the Nemotron post-training v3 collection and built an interactive Embedding Atlas to explore it.

The new @hf.co storage Buckets open up the Hub beyond models and datasets. Example: IIIF image hosting. With Buckets, just upload static tiles and any IIIF viewer zooms straight from CDN!

The new @hf.co storage Buckets open up the Hub beyond models and datasets. Example: IIIF image hosting. With Buckets, just upload static tiles and any IIIF viewer zooms straight from CDN!

74GB of Dutch PDFs, filtered and written back to the Hub - without touching local disk! Hub is your disk! I built a PoC adding sink_parquet for @pola.rs to stream writes to @hf.co's new Storage Buckets via Xet. Constant memory ~18 min on a 2-vCPU machine.

There is no best VLM OCR model - rankings can flip completely by document type. I built ocr-bench: run open OCR models on YOUR documents, get a per-collection leaderboard. VLM-as-judge with Bradley-Terry ELO, all running on @hf.co. No local GPU needed.

Is it worth re-OCR'ing old library index cards? Re-OCR'd 453,000 from @bpl.boston.gov's rare books catalogue. ~$50 compute using @huggingface Jobs BPL's own guide calls their search "extremely unreliable." Does better OCR + semantic search help fix it? Demo space link below

Is it worth re-OCR'ing old library index cards? Re-OCR'd 453,000 from @bpl.boston.gov's rare books catalogue. ~$50 compute using @huggingface Jobs BPL's own guide calls their search "extremely unreliable." Does better OCR + semantic search help fix it? Demo space link below

Ran the same OCR models on 68 pages of historic newspaper. Every model hallucinated or looped. DeepSeek-OCR-2, LightOnOCR-2, GLM-OCR – all melt down on dense newspaper columns. You can try yourself using this @hf.co dataset: huggingface.co/datasets/dav...

Ran the same OCR models on 68 pages of historic newspaper. Every model hallucinated or looped. DeepSeek-OCR-2, LightOnOCR-2, GLM-OCR – all melt down on dense newspaper columns. You can try yourself using this @hf.co dataset: huggingface.co/datasets/dav...

Llama.cpp joins Hugging Face github.com/ggml-org/lla...

Wrote a slightly more detailed guide on how to do this with your own collections/materials: danielvanstrien.xyz/posts/2026/r...

Llama.cpp joins Hugging Face github.com/ggml-org/lla...

Re-OCR'd the complete 1771 Encyclopaedia Britannica (2,724 pages) with a single command on @hf.co Jobs. - 0.9B model (GLM-OCR) ~$0.002/page ~$5 total on an L4 GPU Before (old Tesseract ocr) → After

The uv-scripts/ocr collection now includes 13 models, including GLM-OCR, a 0.9B model that scores 94.6% on OmniDocBench. One command to run any of them on your dataset via @hf.co Jobs. huggingface.co/datasets/uv-...

Join us tomorrow for a demo of IIIF Illustration Detector! Zoom link: iiif.io/community

Datasets and benchmarks drive AI progress, but finding papers that introduce new ones means digging through thousands of arXiv abstracts. Updated the Dataset Papers on ArXiv app to surface them: 52K+ papers classified as introducing new datasets from 212K CS papers.

Join us Feburary 11 for a demo of @danielvanstrien.bsky.social's IIIF Illustration Detector. Zoom on the IIIF Community Calendar: iiif.io/community

Built an object detector from zero-labelled data in one afternoon with help from Claude Code (it can do more than vibe code, TODO apps...) SAM3 on HF Jobs → correct the errors → train YOLO → repeat. Three rounds: 31% → 99% accuracy on historical index cards from @natlibscot.bsky.social

Built an object detector from zero-labelled data in one afternoon with help from Claude Code (it can do more than vibe code, TODO apps...) SAM3 on HF Jobs → correct the errors → train YOLO → repeat. Three rounds: 31% → 99% accuracy on historical index cards from @natlibscot.bsky.social