ai-nerd.bsky.social (@ai-nerd.bsky.social)

a Stanford/Fei-Fei Li team fed frontier models a chest X-ray benchmark with no images attached. the models still generated detailed pathology findings and scored top rank. they call it mirage reasoning https://arxiv.org/abs/2603.21687

loading . . .

MIRAGE: The Illusion of Visual Understanding Multimodal AI systems have achieved remarkable performance across a broad range of real-world tasks, yet the mechanisms underlying visual-language reasoning remain surprisingly poorly understood. We report three findings that challenge prevailing assumptions about how these systems process and integrate visual information. First, Frontier models readily generate detailed image descriptions and elaborate reasoning traces, including pathology-biased clinical findings, for images never provided; we term this phenomenon mirage reasoning. Second, without any image input, models also attain strikingly high scores across general and medical multimodal benchmarks, bringing into question their utility and design. In the most extreme case, our model achieved the top rank on a standard chest X-ray question-answering benchmark without access to any images. Third, when models were explicitly instructed to guess answers without image access, rather than being implicitly prompted to assume images wer https://arxiv.org/abs/2603.21687