Oh wow, deepseek is starting to make serious progress on LLMs that offload memory to external storage: github.com/deepseek-ai/...

ZLUDA, now in its third iteration, has added support for CUDA 13.1 compatibility on non-NVIDIA GPUs (well… AMD GPUs). - 1st iteration: Intel created ZLUDA as a drop-in replacement for CUDA on non-NVIDIA GPUs. - 2nd iteration: AMD took over development after Intel dropped support.

Anthropic announces Claude for Healthcare and expanded Life Sciences tools. By connecting directly to data sources like CMS and Medidata, Opus 4.5 aims to reduce administrative burdens. There are some promising ideas, but we need to see if they can deliver. #MedSky #MLSky

One of my favorite findings: Positional embeddings are just training wheels. They help convergence but hurt long-context generalization. We found that if you simply delete them after pretraining and recalibrate for <1% of the original budget, you unlock massive context windows. Smarter, not harder.

OpenAI's GPT OSS is still insanely underrated as a highly adopted open LLM. Downloads are out of control.

cseweb.ucsd.edu/~tzli/novelt... I gave an internal talk at UCSD last year regarding "novelty" in computer science research. In it I "debunked" some of the myth people seem to have about what is good research in computer science these days. People seemed to like it, so I thought I should share.

NEW: China is testing a megawatt-scale airborne wind system that captures stronger high-altitude winds, potentially producing up to 10× more energy with 40% less material and 30% lower costs. It could also be quickly deployed to supply emergency power after disasters.

Happy arxiv:2601 to all who celebrate! 😉🎉🔬 We are so thankful for everyone who uses arXiv everyday to share and discover new research. Happy new year to all - here's hoping for a 2026 full of #openaccess science & lots of new discoveries.

Seems likely that at some point AI will make it much easier to hack critical infrastructure, create bioweapons, etc etc. Many argue that if so, a hardcore nonproliferation strategy is our only option. Rising Tide launch week post 3/3 is on why I disagree 🧵 helentoner.substack.com/p/nonprolife...

Explaining Human Preferences via Metrics for Structured 3D Reconstruction @jacklangerman.bsky.social Denys Rozumnyi, Yuzhong Huang, @ducha-aiki.bsky.social tl;dr: we asked 3D modelers to rank wireframe reconstructions & compared it to ranking by metrics. Observations🧵 1/ arxiv.org/abs/2503.08208

Struggling with Python `re`? regex is a superset of re with additional functionality and more thorough Unicode support. It is very powerful and still kind of an open secret. When I saw @howard.fm explain this I just had to write about it! Today I Learned: carlo.ai/til/regex-se...

Currently reading a study about influenza aerosol transmission, published in 2020 and shows more RNA copies (log scale) in fine aerosol fraction. In 2020… and we still haven’t sorted out air safety in hospitals, and we are still whinging about flu being such a big problem (which it is).

This is fascinating, so LLMs.txt is essentially a hybrid of a sitemap and schemas, accepted by a fair few AI chatbots. You can reference all manor of information about you and your brand in there - to improve AI interpretations. towardsdatascience.com/llms-txt-414...

Late last year I converted some of my projects to fastht.ml - and have been doing all new projects with it. My productivity and enjoyment has increased significantly. An absolute joy to use. Thousand thanks to @howard.fm et al for the fantastic tooling.

I have to admit that except for the occasional cross-posting, not sure I’ll keep very active on here. There was a big opportunity to migrate the core AI/ML community that was unfortunately squandered with all this hf dataset backlash.

Thanks to @howard.fm for the llms.txt initiative! For context: (Analogy) -> llms.txt is like robots.txt; helps LLMs to provide information by using a website at inference time.

Demis Hassabis, James Manyika, and I wrote up an overview of the AI research work & advances across Google in 2024 (Gemini, NotebookLM, robotics, ML for science, & advances in responsible AI+more). 🎊 Given it a read or paste it into NotebookLM to listen, if you prefer! blog.google/technology/a...

I would like a small non-lethal but painful electrical shock to come through the keyboard of anyone who uses the phrase “do better.”

Getting myself set up here. I found the Sky Follower Bridge Chrome plugin pretty helpful (thanks @kawamataryo.bsky.social!) chromewebstore.google.com/detail/sky-f...

🔥 Host your apps for FREE and simplify your infrastructure! 🚀 Tired of complex cloud hosting and the recurring bills that come with it? Host locally on hardware you already own and access anywhere with a browser. It's simpler than you think! Learn how in my latest post. 🔗👇

General consensus in the replies and quotes of this seems to be that the entire concept of "AI skills" is a joke - how hard is typing text into a chatbot, really? I will continue to argue that it's genuinely difficult, and that the challenge in using these tools is widely underestimated

A new paper dropped from DeepMind: Deliberation in Latent Space via Differentiable Cache Augmentation The trouble is, it's not very readable. I tried making a thread here, but it got far too long, so it's a blog now: timkellogg.me/blog/2024/12...

Wrote up my notes on ModernBERT, the brand new modern alternative to 2018-era BERT released by @benjaminwarner.dev and @howard.fm and team simonwillison.net/2024/Dec/24/...

Are you a Python programmer interested in Quantum Computing? This blog post provides a crossover. Using @howard.fm's "fastcore" library for quantum. I explain what fastcore is, how its Transform functionality works and dive into the quantum use case. (1/2) carlo.ai/posts/fastco...

This week we released ModernBERT, the first encoder to reach SOTA on most common benchmarks across language understanding, retrieval, and code, while running twice as fast as DeBERTaV3 on short context and three times faster than NomicBERT & GTE on long context.

Good morning. Let me check if I’ve got this right. - Juni created a bot that shows what Aaron (head of trust and safety) likes - His likes are public information - Aaron likes a porn post - Trust and safety ban the bot and creator in 16 minutes - Creator appeals and ban is upheld

If you read nothing else about AI this week, read this - absolutely fascinating, ties together a whole bunch of recent developments in the field Wrote my own notes here but really this piece defies summarization, you should absorb the whole thing simonwillison.net/2024/Dec/19/...

(Worse, much of the AI research community reacted to his NeurIPS keynote as if it were an epiphany rather than a repetition of what everyone outside AI has been saying for two years. The deification makes me throw up in my mouth a little.)

New AI Snake Oil essay: Last month the AI industry's narrative suddenly flipped — model scaling is dead, but "inference scaling" is taking over. This has left people outside AI confused. What changed? Is AI capability progress slowing? We look at the evidence. 🧵 www.aisnakeoil.com/p/is-ai-prog...

I'll get straight to the point. We trained 2 new models. Like BERT, but modern. ModernBERT. Not some hypey GenAI thing, but a proper workhorse model, for retrieval, classification, etc. Real practical stuff. It's much faster, more accurate, longer context, and more useful. 🧵

👀something's coming...

i asked Claude to write a Barthelme-esque short story with the aesthetic sensibilities of "The School", and it gave me this. i mean. i mean.

This is number 1 on MTEB Leaderboard, but the VRAM requirement may be higher than its peers since the base model is gemma-2-9b-it.

Wow amazing! An awesome science tool produced by awesome scientists with the help of the awesome nbdev :D

Non-commercial license, as usual from MSR. 🙄 And of course they don't even give you the option to pay for it.

I'm excited to announce @skyseed.fund, the world's first incubator & fund exclusively focused on Bluesky and @atproto.com. We're just getting started but have $1 Million in committed capital, and will be doing developer grants as well as pre-seed/seed funding. www.prweb.com/releases/ope...

AI hype is making AI researchers forget painfully learned lessons core to the field. There's an emerging cope that progress in capabilities isn't slowing down—it's just invisible as benchmarks are saturated; vibe checks are useless because models are now superhuman so we can't perceive improvement.

Dear friends, I'm rejoining Redis. It's a long story, so it deserved a blog post to explain all the details: antirez.com/news/144

@howard.fm falling in love with #fasthtml and htmx. So well done. Before that re-frame for ClojurScript was my reference. You gave the Python community perhaps the last big piece in the puzzle to build robust Python solutions from UI to DB. Thank you.

An updated intro to reinforcement learning by Kevin Murphy: arxiv.org/abs/2412.05265! Like their books, it covers a lot and is quite up to date with modern approaches. It also is pretty unique in coverage, I don't think a lot of this is synthesized anywhere else yet

Hello, world. So I caved and got on Bsky :-) I finally finished my book, AI Engineering, and I'm excited to get back to building. So many fun applications to build! What are you excited about?

Introducing TTS WebGPU: The first ever text-to-speech web app built with WebGPU acceleration! 🔥 High-quality and natural speech generation that runs 100% locally in your browser, powered by OuteTTS and Transformers.js. 🤗 Try it out yourself! Demo + source code below 👇

Super exciting update to nbsanity. I've incorporated @simonwillison.net 's shotscraper. Now, all new renders get a fancy social card! This makes nbsanity a nice microblogging utility Examples of rendered notebooks: 1/3 nbsanity.com/static/6a987...

AI skepticism / criticism would hit a lot harder if the field wasn't full of prominent folks repeatedly claiming AI doesn't work / has no value. Hard to engage with something when it can seemingly support such a weird conclusion

Yes. This experiment will work if you make it your own. Customize your experience, build new ones, make a social network that can be controlled by the people who use it an idea we do not go back on.