I’ve seen great results getting an LLM to write thorough and detailed prompts for another LLM. It’s great to see this area explored in a much more thorough and systematic way.

I LOVE when science is applied to "toddler questions" like "Why is the sky blue?" or "Why is ice slippery?" Because the answers can be so SUPRISING. Let's dig into recent revelations about how ice skates work & why tires skid across black ice. First, we'll dismiss 3 VERY GOOD hypotheses as wrong.

I've been in SF for four days and on each day I've had multiple conversations where people (self included) express deep feelings, fear, wonder and worry about their skills being replaced or (this is less named explicitly but an assumed driver I think) no longer valued by others. Big threats to feel

The better LLMs get, the more data scientists are going to find out the truth.

Must read. I’ve spent much of the past year visiting Big Locals, and have never been more certain that its model would deliver significant results if backed at scale (as the government is doing with Pride in Place) It is remarkable what some places have done given just a little money and time.

Arthur Ganson’s “Machine with Concrete” The magic of gears: the input shaft is driven at 200 rpm, but the output shaft will take over two trillion years to make one turn. So the final gear is embedded in a block of solid concrete. Via @kottke.org makezine.com/article/craf...

"Synthetic Panels" make no sense. The LLM is trained on data from a particular point in time. This is the same as querying the distribution of responses from an LLM with built-in assumptions about what the distribution can look like. Well done surveys are designed for the opposite.

Radiolaria are microscopic unicellular sea creatures that create skeletons from silica. Their skeletons are among the most beautiful and complex natural microstructures, resembling elaborate filigree. blog post: n-e-r-v-o-u-s.com/blog/?p=9861

It's nearly that time again! Join us on Tue 4 Nov for the 101st PyData London meetup. Using Hugging Face's smolagents framework to create a Fantasy Premier League AI Agent - @geirfreysson.bsky.social No More Boilerplate: Data-Driven Testing Made Easy - @feregri.no www.meetup.com/pydata-londo...

Seeing flowers from my new aster for the first time is a real autumn treat!

I recommend this post from @simonwillison.net: simonwillison.net/2025/Oct/7/v... He lists the engineering skills required to use AI-generation most effectively to build quality software, based on his experience using AI on his projects. In short, we can't replace software engineers with vibe coding

It’s nearly time for our 100th Meetup! 🥳 Looking forward to a full roster of lightning talks, reflections from organisers past and present (including many of the OG crew!) and of course well-deserved celebrations--maybe even with cake! 🎂

*deep breath* the fact that these models produce plausible output and also hallucinate (e.g. produce completely novel but reasonable output) has always been why they're so exciting for computational social scientists

Incidentally, my personal favorite interpretation of advancements in computer science is that most of the progress we've made has been in defining the problems themselves. Many successes correlate to clearer definitions of what "good" means or abstracting the task away from weak definitions.

Great to hear Customs House will be redeveloped! Such a grand building in an amazing location by the Thames, but today you have to follow a rickety walkway and peer at it through railings. Big improvement for the Thames Path there. customhouse2024.co.uk www.ianvisits.co.uk/articles/his...

Python: The Documentary | An origin story What began as a side project in the 1990s now powers AI, data science and some of the world's biggest companies. But Python almost disappeared at one point. Featuring Guido van Rossum, Travis Oliphant, Barry Warsaw, and many more! youtu.be/GfH4QL4VqJ0?...

I was particularly pleased to hear Ruby Waller tell us about apprenticeships in data. I had no idea this was a possible route into DS until she told me at the conference this year. So glad she agreed to share her journey!

Had a great PyData London tonight! Was a real treat to hear from @gaelvaroquaux.bsky.social on @skrub-data.bsky.social and the real world data pains its solving. (Try it if you haven’t already; super easy to get going!)

PyData London 2025 keynote: Tony Mears -- Innovation is Dead Innovation gets stifled in the public sector by risk aversion and punishment for failure. But there is hope: Tony shares a way forward, drawing on his experience directing operations in the NHS and UK Space Agency. youtu.be/XMLbJYM46xo

on.ft.com/4oVBaBe My day trip to medieval England Jousting is back in vogue after a 500 year absence. Fantastic pictures too!

Fascinating interview from Odd Lots on what happens when a startup dies. Not often formal bankruptcy. Finding buyers for some company assets can be challenging—like human skulls! podcasts.apple.com/gb/podcast/o...

Excited about this one!

Today at #EuroScipy2025, @glemaitre58.bsky.social and I presented a tutorial on pitfalls of machine learning for imbalanced classification problems. We discussed what (not) to do when fitting a classifier and obtaining degenerate precision or recall values. probabl-ai.github.io/calibration-...

I'll admit, I was skeptical when they said Gemini was just like a bunch of PhDs. But I gotta admit they nailed it.

Fusion energy start-up claims to have cracked alchemy This is really cool: modifying the fusion process could produce gold from mercury almost as a side effect, worth almost as much as the electricity produced each year. on.ft.com/3IXXlWD

Whisper bug report: 'Complete silence is always hallucinated as "ترجمة نانسي قنقر" in Arabic which translates as "Translation by Nancy Qunqar"' social.coop/@brainwane/1... It's fun, but as the thread points out not a new problem, e.g. fanfare -> "subscribe"—and would be fixed in English already.

Visited a few Sculpture in the City exhibits on my afternoon cycle today. Will see if I can find the rest over the next few weeks! www.sculptureinthecity.org.uk

Lovely evening for a cycle yesterday.

Great panel on driving meaningful change to developer experience. "We need better measures than satisfaction that connect directly to things *under our responsibility*, like are people able to access their best problem solving selves, is wellbeing being protected, and quality of problem solving"

Excited the videos are coming out so I can catch up on the talks I wanted to see at the time (like this one) but missed!

Good Science xkcd.com/3101/

Poundland sold for less than a pound on.ft.com/4kWnJOy The article doesn’t say if they went into a shop to complete the transaction, but I choose to believe they did.

Really good keynote from Leanne Fitzpatrick @financialtimes.com at @pydatalondon.bsky.social this morning. AI slop: gen AI is the perfect marketer’s tool. But with a lot of negative externalities!

Welcome to the PyData main conference! 🎉 @jaspajjr.bsky.social getting things going with opening remarks alongside Andy Terrel from @numfocus.bsky.social

LAST CALL! PyData London starts TOMORROW! Looking forward to being part of all the conversations and human connections you just can't get online--so make sure you're in the room. A few last-minute tickets still available if you're quick. See you there! pydata.org/london2025

I’m really* fun at parties, and today a complete stranger made the mistake of telling me they play Civilization, so I broke out one of my favorite topics: 🧠The psychology of randomness in video games *(completely not)

Late spring flowers catching the sun. I wish I could grow these on my balcony!

Some innovative house designs that cater for disabilities and make living there a pleasure, rather than feeling like a clinical and begrudging afterthought. And they're nicer for everyone! The aesthetics of accessibility: design for disability that’s dynamic not dreary on.ft.com/43lRYbH

Just 23 days to the PyData London conference! 🚨 FIRST RELEASE SOLD OUT 🚨 Ready to soak in the knowledge, meet smart folks, discuss AI hot takes, and groan at some questionable data jokes? 6-8 June at Convene Sancroft, St. Paul's Check out the schedule and get tickets here pydata.org/london2025

Published some notes on Microsoft's phi4-reasoning model, an 11GB download (via Ollama) which may well overthink things... it produced 56 sentences of reasoning output in response to my prompt of "hi" simonwillison.net/2025/May/6/p...

One of my very favorite things about consulting with engineering organizations, for all the difficulties we face in our tech cultures, is that so many people with such different backgrounds from my own, share my value of wanting to be in the reality game. I truly believe many of us want to be there.

The PyData Global 2024 videos are now up on YouTube! To kick things off with a bang, here's Wes McKinney's talk on Retooling for a Smaller Data Era (He had connectivity issues; talk starts 7:45) youtu.be/w4aYrav8-zE?...

After last year's sellout, PyData London 2025 is happening 6-8 June at Convene Sancroft, St Paul's. Join us for 3 days of tutorials, talks and the chance to meet and hang out with a great group of data enthusiasts! Tickets already selling fast--get yours here today! pydata.org/london2025

Kicking off tonight’s PyData London with @johnsandall.bsky.social telling us about Death by RMSE: Cautionary Tales of Metrics Gone Wild Choosing the wrong eval metrics can look great but give terrible real world performance—it’s always a trade off!

If you're a fan of Isochrone maps and US census data (and who isn't?) Dan Snow just released opentimes.org and it's spectacular - I published some notes about it here, there's some very clever data engineering going on under the hood simonwillison.net/2025/Mar/17/...

*Please repost* @sjgreenwood.bsky.social and I just launched a new personalized feed (*please pin*) that we hope will become a "must use" for #academicsky. The feed shows posts about papers filtered by *your* follower network. It's become my default Bluesky experience bsky.app/profile/pape...

The new GOV.UK login system is the first step in a radical transformation of the British state (FREE TO READ!) takes.jamesomalley.co.uk/p/login-to-a...

DEADLINE TODAY(23:59 GMT) to get your talk or tutorial in for PyData London 2025! Register as a speaker here pydata.org/london2025/cfp

Running Pong in 240 Browser Tabs. “That’s 240 browser tabs in a tight 8x30 grid. And they’re running pong!” [eieio.games]