I've been meaning to write a blog post a bit like this one for ages, but this is much better than I would have done. I think the author is right, private evals are very important and this post gives a good framework of how to design your own -> thundergolfer.com/blog/private...

Always really like Alex's post and I feel like this one nails it again: "the dynamic that matters is not a timeline of capacities but a timeline of accuracies" if you're not already maxed out with AI predictions then this one is worth a read

Firebase studio looks quite fun butttt: "To block the use of your prompts and responses for model training, do not use the App Prototyping agent, and do not use Gemini in Firebase within Firebase Studio. To block the use of your code for model training, turn off code completion and code indexing..."

Blog post to try and tie together a few themes I've been reading about this week -> tomhipwell.co/blog/cursor_...

Not sure if this is obvious but probably the most important evaluation criteria for any AI tool at the moment is the ability to choose your own model (and bring your own API key or OpenAI compatible API)

This is a great post, worth reading. I re-blogged it here to try and summarise the reasoning -> tomhipwell.co/blog/the_mod...

Claude 3.7 Sonnet

Product teams talking too much about ICPs is a red flag. ICPs are for sales and marketing teams. They need narrow focus to max win rate. Product teams need to understand ICP for prioritisation, but think in PMF strength across segments. Peripheral vision. This is how you expand PMF and win more.

Another day, another AI dev flow. There’s some common patterns emerging now (using markdown files like spec.md etc.). This blog gives a step by step guide and prompts to borrow. The advice reduces to “spend a lot of time planning with reasoning models up front” -> harper.blog/2025/02/16/m...

Will GenAI mean the end of software engineering? Really thoughtful take from @chiphuyen.bsky.social in The Pragmatic Engineer Podcast: Perhaps software engineering will change, like writing changed hundreds of years ago thanks to printing Full: www.youtube.com/watch?v=98o_...

Really enjoyed Dario's blog post, I thought it had a bunch of interesting, verifiable predictions (I'm less interested in the export control stuff). It'll be interesting to see if they come good over the course of '25. Worth a read, cuts through the noise -> darioamodei.com/on-deepseek-...

Great read, useful survey of the field.

Good list, would be good to hear additions from #dataBS, I would add "what are embeddings" by @vickiboykis.com -> www.latent.space/p/2025-papers

I like @simonwillison.net definition of slop, but we're missing a word to describe an algorithm that pushes low grade content. I'd propose gruel, e.g my LinkedIn feed is all gruel now. Netflix recommends gruel all the time. Spotify's playlists are stuffed full of gruel. You get the picture.

If you can solve this puzzle, then do not despair! You are still much smarter than o3

Great post, a starting point for something that has been confusing for a while

webdev arena is a _really_ fast way to get a feel for the coding abilities of the different models out there. Worth five minutes of your time to design a tricky prompt, then quickly assess each model generation as they get released -> web.lmarena.ai

Interesting ideas

I've had this post sat in my drafts for well over 6 months now, but with the release of Sora to GA yesterday I thought I'd share it - Sora: An Idiot's Guide. tomhipwell.co/blog/sora/

Reinforcement Learning: An Overview This manuscript gives a big-picture, up-to-date overview of the field of (deep) reinforcement learning and sequential decision making, covering value-based RL, policy-gradient methods, model-based methods, and various other topics. arxiv.org/abs/2412.05265

I had a play about with genini-1206 last night, trying some prompts into both o1 and 1206 in parallel, both models are crackers. Vibes for me are better with o1, feels a bit more succinct and gets to the point quicker. Early days of course.

Alibaba Qwen team just released the base models for Qwen2-VL. You can also wait for them to release Qwen2.5-VL, which should be sooner than later. 2B: huggingface.co/Qwen/Qwen2-V... 7B: huggingface.co/Qwen/Qwen2-V... 72B: huggingface.co/Qwen/Qwen2-V...

A test of how seriously your firm is taking AI: when o-1 (& the new Gemini model) came out this week, were there assigned folks who immediately ran the model through your internal, validated, firm-specific benchmarks to see how useful it as? Did you update any plans or goals as a result?

Some great ideas here, worth your time to explore

First new post in a couple of weeks! There's been a lot of activity around regattastorage.com this week, so I decided to write about the space. tl;dr It's pretty exciting!

This is nuts 🥜🐿️🐿️🤯

Insightful piece from @marcbrooker.bsky.social on Aurora DSQL. Aurora is a "distributed sql" db with SQL and ACID, global active-active, scalability both up and down (independent scaling of compute, reads, writes, and storage), and Postgres compatibility (psql works!)-> brooker.co.za/blog/2024/12...

Super handy index of publicly available LLM implementations / reference architectures, all searchable in one place, thanks to @strickvl.bsky.social -> zenml.io/llmops-datab...

I wrote a fast guide on getting up and running quickly (an hour or so) and cheaply (free!) with semantic search for toy use cases (blogs etc.). I used chromadb, which is an Apache 2.0 licensed sqllite wrapper; all-MiniLM-L6-v2 is used for embedding generation -> tomhipwell.co/blog/baked_s...

Did you know that attention across the whole input span was inspired by the time-negating alien language in Arrival? Crazy anecdote from the latest Hard Fork podcast (by @kevinroose.com and @caseynewton.bsky.social). HT nwbrownboi on Threads for the lead.

To showcase how much you can do with just a 1.7B LLM, you pass free text, define a schema of parsing the text into a GitHub issue (title, description, categories, tags, etc) - Let MLC & XGrammar do the rest! That's it, the code is super readable, try it out today! 🤗 huggingface.co/spaces/reach...

This S1 analysis from Meritech went viral due to the (compounding!) IPO ratchet that ServiceTitan are subject to. About halfway down there are some handy benchmarks for median/top decile pre-IPO performance in vertical SaaS -> www.meritechcapital.com/blog/service...

End to end tutorial of function calling with Llama-3.2-3B-Instruct, building gradually from string templating, to using Jinja, to implementing web search with Brave, via @sergiopaniego.bsky.social -> github.com/huggingface/...

Sharing a timelapse of a small fleet of Figure 02 humanoid robots doing work Our robots are reasoning fully autonomously Much of the core engineering infrastructure has been laid the last two years, 2025 is going to be 🤯

Feels good to be mentioned on HN for engineers learning AI 🥰 Helping others is a big reason I write. Here's a list on ML/AI: ## Building AI systems • Patterns for Building LLM-based Systems: eugeneyan.com/writing/llm-... • What We’ve Learned From A Year of Building with LLMs: applied-llms.org

uv cheat sheet, handy intro to the tool if, like me, you've not tried it yet-> docs.google.com/document/d/1... via @brandonrohrer.com

Bought a copy of "The Little Book" and I've been keeping it on my desk for quick reference, highly recommended!

Using LLMs for query or document expansion in retrieval (e.g. HyDE and Doc2Query) have scores going 📈 But do these approaches work for all IR models and for different types of distribution shifts? Turns out its actually more 📉 🚨 📝 (arxiv soon): orionweller.github.io/assets/pdf/L...

The AWS Lambda PR/FAQ has lots of small, golden nuggets. PR/FAQs like this are a bit wordy but you can see how effectively the technique is used here to explain the impact of not just a new architecture but also a shift in AWS’s compute billing model - www.allthingsdistributed.com/2024/11/aws-...