Vibe-coding’s complement is unit testing. If the writing of code is commodified, the validation of output becomes scarce.

Randomized trial AI for legal work finds Reasoning models are a big deal: Law students using o1-preview had the quality of work on most tasks increase (up to 28%) & time savings of 12-28% There were a few hallucinations, but a RAG-based AI with access to legal material reduced those to human level

🚨 New working paper with Caitlin Gorback! We ask what happens when households are more likely to WANT to own a home for cultural reasons? We find homeownership increases, they're more responsive to credit supply shocks, and more of their retirement portfolios are in real estate. 🧵

A new generation of AIs: Claude 3.7 and Grok 3 https://www.oneusefulthing.org/p/a-new-generation-of-ais-claude-37 #AI #Claude #Grok

A professional workflow for translation using LLMs - Simon Willison. https://simonwillison.net/2025/Feb/2/workflow-for-translation/

🧪 A @ai.nejm.org study found that GPT-4 translation of discharge notes significantly improved patient understanding, especially for Black, Hispanic, and older patients. AI-driven readability could help close health literacy gaps. 🩺💻 tagging: @downing.bsky.social @thelizarmy.bsky.social

I am worried LLM researchers sometimes bury the lede with regard to "should we trust these systems". Framing below is: LLMs are failing to "earn human trust". But it turns out it's the humans who cannot be trusted - even seeing the LLM's answer, the humans do worse than the LLM!

I think one way of describing these results to our future LLM overlords is “inference time compute scaling leads to thinking more but not to thinking better - in humans”

Generative AI has flaws and biases, and there is a tendency for academics to fix on that (85% of equity LLM papers focus on harms)… …yet in many ways LLMs are uniquely powerful among new technologies for helping people equitably in education and healthcare. We need an urgent focus on how to do that

Let me try to formalize some thoughts about Gen AI adoption that I have had, which I will call "The Bedazzlement Curve". Most people still underestimate how useful Gen AI tools would be if they tried to find use cases and overestimate the issues - they're in "The Valley of Stochastic Parrots".

Reporting on AI adoption rates tends to show the importance of having priors.

In many examples of people actually implementing Gen. AI-based workflows, building the automation requires experience with the task at hand - suggesting that there might be upskilling / demand for experienced workers in those areas at least in the short-medium term, rather than simple “replacement”

The new Deep Research feature from Google feels like one of the most appropriately "Google-y" uses of AI to date, and is quite impressive. I've had access for a bit and it does very good initial reports on almost any topic. The paywalls around academic sources puts some limits around it, though.

Was very surprised to stumble across a graph from my own research in a presentation by Benedict Evans today! He makes the fair point that predicting technology effects is hard! Although I prefer to call our analysis "bottom-up" as it builds from microdata to a firm level exposure measure. 1/2

It's incredibly encouraging that even models for analytical purposes, like o1, can recite Shakespeare. This means that there are still many "storage" parameters not fine-tuned for analytics and means that distillation can get large performance improvements at smaller model size.

I had to tell a Ph.D. student today what o1 is. "The future is already here, it's just not evenly distributed"

I found this Stratechery framework for how technologies evolve by the interplay of hardware, input modes, and applications quite thought-provoking - with innovations in one triggering innovations in the other dimensions. Science seems to progress in similar ways. stratechery.com/2024/the-gen...

I think firms worrying about AI hallucination should consider some questions: 1) How vital is 100% accuracy on a task? 2) How accurate is AI? 3) How accurate is the human who would do it? 4) How do you know 2 & 3? 5) How do you deal with the fact that humans are not 100%? Not all tasks are the same.

how do researchers use LMs in their work & why? we surveyed 800 researchers across fields of study, race, gender, seniority asking their opinions on: 🐟 which research activities (eg coding, writing) 🐠 benefits vs risks 🦈 willingness to disclose findings in @simonaliao.bsky.social's thread 🧵

How can managers identify GenAI use cases? I was struggling to find a good framework to teach my MBA students how to find GenAI use cases in their orgs - so I made my own! I called it the "BEAST" framework for finding LLM use cases - see details below.

Happy ChatGPT Day to those who celebrate it! This birthday AI is being very demure, very mindful, even on its special day.

The batch API pricing for major LLMs seems like great price discrimination in favor of academic use cases and I am not complaining.

Does Generative AI upskill or downskill the jobs that are affected? One perspective (not necessarily the correct one!) is that of the "paradox of automation" where workers need MORE training and specialized skills as those tasks that cannot be automated are the specialized ones...

One good use case of LLMs for research that I have found: rapidly going deeper on existing literature reviews to find the papers most relevant to me. Steps: 1. Paste in lit review 2. Ask for web search on all the papers, to get titles and abstracts => Saves me lots of separate searches

The actual usage pattern of Generative AI look very different from a simple "automation = bad for workers" perspective: firms often use Gen. AI to ENHANCE worker capabilities - we should be interested in how this leads to a restructuring of workplaces and the assignment of different tasks to jobs.

This is a fascinating study, and speaks to the vast perception gap between what LLMs are capable of and what people think they are capable of. Doctors using ChatGPT for diagnosis performed *worse* than ChatGPT alone, because they kept second-guessing the model. www.nytimes.com/2024/11/17/h...

For Bluesky-Curious Econ Lovers, a Quick Start guide to plugging into the economics community here. It aims to lower the costs & boost the benefits for folks to engage here. If it seems useful, please share it here & especially on X. 📉📈 #️⃣#️⃣ aaronsojourner.org/for-bluesky-...