A richly annotated dataset of co-speech hand gestures across diverse speaker contexts www.nature.com/articles/s41... Dataset comprising 2373 annotated gestures, 9 speakers across 3 distinct categories: University lecturers, Politicians, and Psychotherapists can be accessed at doi.org/10.17605/OSF...

Applications are now open for the MARCS International Visiting Scholar Program 2026! 🌏 We are pleased to offer scholarships for PhD students and postdocs for visits of 1–3 months before the end of 2026. 📅 Applications close 4 December 2025. If you are interested, email [email protected]

Distinct Portions of Superior Temporal Sulcus Combine Auditory Representations with Different Visual Streams www.jneurosci.org/content/45/4... Analysed open-source auditory cortex fMRI data from people watching a movie with ANNs to investigate STS & 2 visual streams

When brain talks back to the eye "The state of our brain shapes what we see, but how early in the visual system does this start? A new study in PLOS Biology shows that brain state-dependent release of histamine modulates the very first stage of vision in the retina" journals.plos.org/plosbiology/...

Here's an interesting new study exploring whether LLMs are able to understand the narrative sequencing of comics and... even the best AI models are *terrible* at it for pretty much all tasks that were analyzed aclanthology.org/2025.finding...

Can you feel what I am saying? Speech-based vibrotactile stimulation enhances the cortical tracking of attended speech in a multi-talker background https://www.biorxiv.org/content/10.1101/2025.10.31.685484v1

Audiovisual Synchrony in Left-hemisphere Brain-lesioned Individuals with Aphasia direct.mit.edu/jocn/article... Found a statistically significant effect of aphasia type on measures of AV synchrony; an effect not explained by lesion volume...damage to left posterior temporal bad for AV processing.

Expectation-driven shifts in perception and production pubs.aip.org/asa/jasa/art... Failed to find evidence that individuals' expectation-driven shifts in perception correlate with those in production ...

Yes, the McGurk effect, that's right -> "The influence of age, listener sex, and speaker sex on the McGurk effect" journals.sagepub.com/doi/10.1177/... Are reports of higher sensitivity to the McGurk effect in females than males influenced by the match of Listener-Speaker sex?

Audiovisual speech perception in Mandarin cochlear implant users across age and listening conditions www.sciencedirect.com/science/arti... "AV cues play a critical role in speech perception for Mandarin-speaking CI users, especially under acoustically challenging conditions"

Visual induction of spatial release from masking during speech perception in noise pubs.aip.org/asa/jel/arti... "There was no enhancement of auditory SRM through visual spatial separation" (shown before) - It did have a negative effect though i.e., to "disrupt existing auditory SRM" ...

Attentional engagement with target and distractor streams predicts speech comprehension in multitalker environments https://pubmed.ncbi.nlm.nih.gov/41136338/

Visible Pre-acoustic Lip Motion Aids Listeners’ Judgments of Speech Onset Times www.researchgate.net/profile/Pete... Visible lip motion facilitates detecting acoustic speech onset ....

Ok, yes, sometimes I post just because of the title Oh, multimodality where art thou? Raised eyebrows in the constructional network www.jbe-platform.com/content/jour... Argues that knowledge about Tell Me About It & raised eyebrows is structured as a nested network of uni- & multimodal constructions

The Influence of Facial Speech Phenomenon on the Learning Process of Children With Dyslexia: Aspects of Susceptibility & Dependency on Visual & Phonological Stimuli pubs.asha.org/doi/10.1044/... Investigated impact of different training programs (phonetic & visual) on learning (ps < .01 & .05, hmm)

Attentional disengagement during external and internal distractions reduces neural speech tracking in background noise https://www.biorxiv.org/content/10.1101/2025.10.17.683146v1

Exploring the McGurk Effect in Cochlear-Implant Users: A Systematic Review brill.com/view/journal... Systematic review carried out using the McGurk paradigm to understand the speech perception mechanism in CI-fitted individuals

sEEG Reveals Neural Signatures of Multisensory Integration in the Human Superior Temporal Sulcus during AV Speech Perception www.jneurosci.org/content/45/4... Used sEEG to directly record from the STG & STS in 42 epilepsy patients >STS takes the lead (i.e., faster that STF) in AV speech perception

Evaluating the temporal order of motor and auditory systems in speech production using intracranial EEG https://pubmed.ncbi.nlm.nih.gov/41062786/

"I would be so entertained if I found out an AI lab had wasted their time cheating on my dumb benchmark!" 😆@simonwillison.net

Mapping the task-general and task-specific neural correlates of speech production: Meta-analysis and fMRI direct comparisons of category fluency and picture naming https://pubmed.ncbi.nlm.nih.gov/41030322/

Check out our fantastic new EnvisionBOX intro video made by Markus Küpper! This video and nomination really feels like another step in professionalizing the platform and making it even more visible, and so hopefully becoming useful to even more people. Thanks everyone for the nominations!

Metacognitive Awareness of Lipreading Gains in Young and Older Adults pubs.asha.org/doi/10.1044/... Investigated the extent older & younger adults are metacognitively aware of AV benefits, focused on: 1) performance improvements 2) changes in confidence & listening effort 3) internal evaluations

The Shared Book Reading Corpus: An audiovisual collection of 44 multimodal caregiver-infant interactions www.nature.com/articles/s41... A collection of AV recordings of English-speaking caregivers and 13–14-month-old infants (N = 44 dyads) reading together in a controlled lab setting

Multisensory assessment for hearing phenotypes www.sciencedirect.com/science/arti... "these findings suggest that visual performance and visual sensory weighting—more than AV integration per se—are modulated by hearing loss"

Visual contributions to the perception of speech in noise https://www.biorxiv.org/content/10.1101/2025.09.26.678636v1

How is valence computed in the brain? Check out our new preprint about a single cell that integrates excitatory and inhibitory input across modalities according to valence and impacts behavioral decisions. An exciting collaboration across many labs. Enjoy reading! www.biorxiv.org/content/10.1...

I'm thrilled to share that our new paper is now published in Psychophysiology (Open Access): onlinelibrary.wiley.com/doi/10.1111/... @zacndr.bsky.social @teamlabuda.bsky.social @fraferri.bsky.social @danlikesbrains.bsky.social

Temporal integration in human auditory cortex is predominantly yoked to absolute time www.nature.com/articles/s41... There's a difference between integrating across absolute time & structure (say) phonemes. Do cortical computations reflect time or structure? Results showed time-yoked computations ⏱️

Distinct Cortical Populations Drive Multisensory Modulation of Segregated Auditory Sources www.jneurosci.org/content/45/3... "findings suggest .. modulatory multisensory behavior is supported by nSTRF neurons and identifies...a functional differentiation between the role that STRF and nSTRF neurons"

The rhythm of sensory input shapes AV temporal processing bpspsychub.onlinelibrary.wiley.com/doi/10.1111/... Expts. used simultaneity judgement tasks & pulsing AV streams amplitude-modulated according to different regular frequencies or following purely rhythmic vs. quasi-rhythmic envelopes.

Measuring the dual-task costs of audiovisual speech processing across levels of background noise. psycnet.apa.org/doiLanding?d... 4 experiments evaluated how AV speech affects dual-task costs during speech perception - reduce effort in difficult listing conditions but increase it in easier ones?

Large-scale color biases in the retinotopic functional architecture are region specific and shared across human brains www.jneurosci.org/content/earl... "predicted what color someone is seeing based on their brain activity, using only knowledge of color responses from other observers' brains"

Disentanglement of prosodic meaning: Toward a framework for the analysis of nonverbal information in speech https://pubmed.ncbi.nlm.nih.gov/40938709/

Mouth and facial informativeness norms for 2276 English words link.springer.com/article/10.3... Mouth & facial informativeness for words, i.e., how visually informative words are based on their corresponding mouth & facial movements - norms are accessible via Open Science Framework (osf.io/mna8j/)

Prior expectations guide multisensory integration during face-to-face communication journals.plos.org/ploscompbiol... Is there a stronger prior tendency to integrate audiovisual signals that convey the same communicative intention (& is this based on an optimal Bayesian decision)?

| | | Bridging | | speech | | and. | | sight. | | ... academic.oup.com/braincomms/a... Ticker-tape synaesthesia - 17 synesthetes & 17 controls diffusion-weighted MRI -> ticker-tapers ⬆️ white matter underlying the SMG + connecting SMG & mid & posterior STG

Beat gestures facilitate lexical access in constraining sentence contexts psycnet.apa.org/doiLanding?d... Tested if beat gestures, visual prominence cues, can aid lexical access (RT) even in situations where other cues (semantics) are supportive of word recognition & if alignment with stress matters

The tracking umbrella: Diverse interpretations under a common neural term nyaspubs.onlinelibrary.wiley.com/doi/10.1111/... Review critically examines neural tracking from both theoretical and methodological perspectives, highlighting how its interpretation varies across studies & how it is measured

Neural entrainment by speech in human auditory cortex revealed by intracranial recordings www.sciencedirect.com/science/arti... 12 people had intracranial electrodes & listened to natural speech oscillatory -> speech edge induced activity persisted after evoked responses thus ...neural entrainment?

Stereoelectroencephalography reveals neural signatures of multisensory integration in the human superior temporal sulcus during audiovisual speech perception www.jneurosci.org/content/earl... Directly recorded from the STG & STS in 42 epilepsy patients who identified single words in AO, VO & AV

Insights From Language-Trained Apes... onlinelibrary.wiley.com/doi/10.1002/... Reviews how studying the brains of language-trained apes might tease apart the relative contribution of evolved species differences, behaviour & environment & how this research uncovers experience-dependent plasticity 🐒

GEPPETO-OFC: An optimal feedback speech motor control model integrating biomechanical constraints and multisensory goal specification https://pubmed.ncbi.nlm.nih.gov/40910954/

Deliberate synchronization of speech and gesture: Effects of neurodiversity and development https://pubmed.ncbi.nlm.nih.gov/40896637/

Neural speech tracking in noise reflects the opposing influence of SNR on intelligibility and attentional effort direct.mit.edu/imag/article... How do SNR, speech intelligibility & attentional engagement relate to each other? How to find out? EEG + eye-tracking + cocktail party with neural tracking

Auditory-motor adaptation and de-adaptation for speech depend more on time in the new environment than on the amount of practice www.nature.com/articles/s44... "data demonstrate that both auditory-motor adaptation and de-adaptation depend more on exposure time than amount of practice"

Parents’ multimodal spatial language structures infants’ in-the-moment attention during spatial play. psycnet.apa.org/doiLanding?d... "how the content of caregivers’ speech, and not simply the presence of speech, along with deictic gestures may shape infants’ attention in real time"

Integration of Audiovisual Motion in Dorsolateral Prefrontal Cortical Neurons journals.physiology.org/doi/abs/10.1... What is the role of the properties of prefrontal neurons in multisensory processing? Dorsolateral prefrontal neurons integrate auditory & visual motion signals in rhesus macaques

Multisensory Stroop effects in emotional speech perception: Age-related changes and cognitive links https://pubmed.ncbi.nlm.nih.gov/40875343/

Does gesture follow speech in describing metaphorical motion events over developmental time? https://pubmed.ncbi.nlm.nih.gov/40811893/