[Weekend Read] DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents - arxiv.org/pdf/2506.11763 The best deep research AI agents have yet to cross the 50% mark in terms of comprehensiveness and depth . They also exhibit a 20% citation hallucination rate. #AI #Research #search

[Weekend Read] Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of Artificial Intelligence - lnkd.in/gnBWstn7 Early impact of #AI on #employment seems to indicate that entry-level white-collar jobs including software engineering, marketing, and support are the most affected.

[Weekend Read] How Much Knowledge Can You Pack into a LoRA Adapter without Harming LLM? arxiv.org/abs/2502.14502 TL;DR:Don't use LoRA to add knowledge to LLMs. Full research note and previous weekend read papers: notes.elie.net/Papers+revie... #AI #LLM #LoRa

[Weekend Read] Subliminal Learning Language models transmit behavioral traits via hidden signals in data arxiv.org/abs/2507.14805 Surprisingly during knowledge distillation, student models unconsciously acquire teacher model characteristics even when training on unrelated data #AI #LLM #RLM

Happy to announce that we open sourced LMEval, a large model evaluation framework purposely built to accurately and efficiently compare how models from various providers perform across benchmark datasets opensource.googleblog.com/2... #AI #LLM #OSS

The leaderboard illusion: arxiv.org/abs/2504.20879 Look at some of the shortcomings of the Chatbot arena which has emerged as the go-to leaderboard for ranking the most capable #AI . Recently Meta abused some of them to game #Llama 4 Behemoth results - sherwood.news/tech/meta-scr...

The Phare Benchmark results key insights include that popularity on benchmarks like LMArena doesn't guarantee factual reliability and that the more confidently a user phrase its query the less willing models are willing to refute controversial claims (sycophancy) - www.giskard.ai/knowledge/go...

[Weekend Read] Exploring LLM Reasoning Through Controlled Prompt Variations - arxiv.org/abs/2504.02111 Show how critical it is to have only relevant data in the model context. Accurately filtering out data is very difficult and simply relying on vector search is NOT the answer. #AI #RAG

[Weekend Read] RealHarm: A Collection of Real-World Language Model Application Failures arxiv.org/abs/2504.10277 This paper by looking at real world examples of AI failure highlight the disconnect between what safety filters block and what goes wrong in practice. #safety #research #ai

[Weekend read] Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verification - arxiv.org/pdf/2502.01839 If you are interested in understanding how scaling computation at generation time help improve model performance this is the paper to read.

[Weekend Read] Measuring AI Ability to Complete Long Tasks - arxiv.org/pdf/2503.14499 The ability of models to perform longer and longer tasks roughly double every 7 months. That's encouraging however I am unsure how true this hold at 99% success rate which is needed to trust agents. #AI #LLM

Accelerating Large-Scale Test Migration with LLMs - medium.com/airbnb-engineeri... Airbnb was able to leverage AI to reduce migration time by about 90% (6weeks / 1.5y) #AI #airbnb

[Weekend Read] AI Search Has A Citation Problem - www.cjr.org/tow_center/we-c... TL;DR: Doing an agent is one thing, getting to the level of reliability where an agent can be trusted is a total different ball game. Evaluating reliability is critical to true progress. #AI #Agent #research

#Gemma 3 is here! Our new open models are incredibly efficient - the largest 27B model runs on just one H100 GPU. You'd need at least 10x the compute to get similar performance from other models 👇 #AI #LLM

[Weekend Read] Reasoning Language Models: A Blueprint - arxiv.org/abs/2501.11223 All you need to know on how thinking/reasoning models are trained and evaluated. #AI #RLM #LLM

[Weekend Read] The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers - www.microsoft.com/en-us/res... The more people are confident in #GenAI the less they think critically. #Research #AI #education

[Weekend Read] SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training - arxiv.org/pdf/2501.17161 - Using Reinforcement Learning (RL) help generalizing and SFT help stabilizing. #AI #LLM #research #RL

[Weekend Read] Humanity’s Last Exam - static.scale.com/uploads/65... New large scale knowledge benchmark where the best models barely reach 9% with DeepSeek-R1 outperforming everyone. #benchmark #research #AI #LLM #deepseek #openai #anthropic #gemini

[Tool Tuesday] LLM the best CLI utility for interacting with Large Models - https://github.com/simonw/llm This comprehensive tool support images, local/remote models and shell workflow. You can for example type: cat https://mycode.py | llm -s "Explain this code" ##LLM ##tool ##AI

[weekend read] Human Creativity in the Age of LLMs - https://arxiv.org/abs/2410.03703 - Worryingly this study shows that ##AI might boost short-term creativity at the expense of long-term one. Figuring out how to leverage ##LLM without degrading human long-term capabilities is a very pressing issue.

[Weekend Read] NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review : https://arxiv.org/pdf/2210.00379 - NeRF models allow to synthetize and render a 3D scene from all direction based of 2D images. This paper is a good summary of this must know type of model. ##AI ##research ##3D

[Weekend Read] Scaling Retrieval-Based Language Models with a Trillion-Token Datastore - https://arxiv.org/abs/2407.12854 Good baseline experiment that put RAG lift at about 8% on knowledge tasks with smaller models benefiting more from it. ##LLM # LLama ##research ##AI ##RAG

[Tool Tuesday] RagFlow: Open-source ##RAG retrieval system - https://github.com/infiniflow/ragflow It as a nice UI, is easy to deploy and implement a lot of novel techniques such as graphrag. Great out of box system if you want to have a chatbot that leverage specialized data ##LLM ##AI ##OSS

[Weekend Read] Fairness Definitions in Language Models Explained - https://arxiv.org/pdf/2407.18454 This paper do a great job at explaining simply the key ideas on how to evaluate ##LLM ##fairness and provide key references. Very useful read if you are interested in the topic. ##AI ##Research

[Weekend Read] Adversaries Can Misuse Combinations of Safe Models - https://arxiv.org/abs/2406.14595?utm_source=bluesky&utm_campaign=elie Study how to combine frontiers models with weaker ones to complete dangerous tasks. ##LLM ##AI ##Research ##Cybersecurity

[Friday Fun] The Vanity Fair No. 41 playing cards deck is the first transformation deck manufactured by The United States Playing Card Company in 1895 https://etteilla.org/en/deck/63/vanity-fair-no-41 ##playingcards ##art ##history

Interesting write-up on how relying on LLM to write code can be very costly https://web.archive.org/web/20240610032818/https://asim.bearblog.dev/how-a-single-chatgpt-mistake-cost-us-10000/ ##LLM ##AI

[Weekend Read] A Careful Examination of Large Language Model Performance on Grade School Arithmetic: http://arxiv.org/abs/2405.00332 By creating from scratch a new math benchmark (GSM1K) the authors show that many models training data are likely polluted with benchmark data. ##AI ##LLM ##GPT

[Weekend Read] Better & Faster Large Language Models via Multi-token Prediction - arxiv.org/abs/2404.19737 Quite an interesting technique to increase LLM performance and speed: predict multiple tokens instead of just the next one. #AI #Research #LLM

[Weekend Read] MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training - arxiv.org/abs/2403.09611 this paper study the factors affecting Multimodal LLM performance including what I think is the best analysis of how data mixture greatly affect performance. #AI #Research #LLM #VLM

[Friday Fun] Jeu du Destin Antique by Eugène D’Auriac etteilla.org/en/deck/15/j... - one of the most beautiful oracle deck the foundation own. Sadly I never found the original one (yet!) #tarot #oracle

28% of the private companies lost their unicorn ($1B+ valuation) status according to new research - blog.equityzen.com/the-state-of... Is the pandemic bubble popping? #startups #vc #tech

[Friday Fun] Original Etteilla divination Tarot "Le Jeu de Toth" etteilla.org/en/deck/10/o... Based on the cards marking this version is likely the original version of the tarot deck invented by Etteilla the first professional Tarot fortune teller. #tarot #divination #art

New paper shows that humans are bad telling apart fact from opinion: 64% avg accuracy. I was curious how #LLM would do so ran an evaluation - All model succeed perfectly except for the earth ambiguous question. Original article misinforeview.hks.harvard.edu/article/fact... #AI #misinformation

[Weekend Read] TacticAI: an AI assistant for football tactics - www.nature.com/articles/s41... This paper is a great reminder that AI is not only about LLM and that specialized models are very useful for a wide variety of tasks. #AI #Research #Football #LLM #DeepLearning

[Friday Fun] This is the original edition of the Waite-Smith #tarot deck - etteilla.org/en/deck/81/o... It is the most well know tarot deck in the world. Now carefully preserved and made available online by our nonprofit dedicated to playing cards the Etteilla Foundation

[Weekend Read] What Was Your Prompt? A Remote Keylogging Attack on AI Assistants - drive.google.com/file/d/1Fk96... MitM side-channel attacks against LLM network traffic that probabilistically recover conversations from various LLMs chatbot despite been encrypted. #AI #cybersecurity #research

[Weekend read] Propagating Universal Perturbations to Attack Large Language Model Guard-Rails - arxiv.org/abs/2402.15911 TL;DR: demonstrate that adding a Guard Rail model doesn't work as well as previously thought to protect LLMs. Bad news for all the startups? #AI #cybersercurity #LLM #research

Google is partnering with Technical University of Munich to advance AI cybersecurity capabilities - www.tum.de/en/news-and-... #ai #cybersecurity

In partnership with NGIS Australia Google is using AI and Google Earth data to find and help restore the Australia’s giant kelp forests - blog.google/intl/en-au/c... Leveraging vision classifiers to map at scale is really one of the most powerful and under-rated application of #AI

🧞 Genie: Generative Interactive Environments - sites.google.com/view/genie-2... a new foundational world model trained from Internet videos that can generate an endless variety of playable (action-controllable) worlds from synthetic images, photographs, and even sketches. #gamedev #AI #deepmind

[Weekend Read] BitDelta: Your Fine-Tune May Only Be Worth One Bit - arxiv.org/pdf/2402.101... Fascinating research that look at how much information LLM fine-tuning add compared to foundational training. The surprising answer is probably close to only 1bit. #AI #LLM #Research

[Weekend Read] New (and first?) PoC attack against mixture of experts (MoE) models which are now a staple of scaling large models - arxiv.org/abs/2402.05526 It's just a proof of concept for now but definitively a direction worth exploring. Glad to see our team exploring this direction. #AI

Gemini is now available on Android and iOS! www.youtube.com/watch?v=b5Fh... #assistant #AI #Google #Gemini #LLM #GPT