"artificial general cleverness" 😆https://mathstodon.xyz/@tao/115722360006034040

Delta-band cortical speech tracking predicts audiovisual speech-in-noise benefit from natural and simplified visual cues www.sciencedirect.com/science/arti... Degraded face vids -> some gains in AV speech comprehension; animations of speech envelope not integrated with auditory speech; Δ band = gain

The Scope and Limits of Iconic Prosody: Head Angle Predicts f0 Changes While Object Size Effects Are Absent direct.mit.edu/opmi/article... Looked at the relationship between head angle & f0 in iconic prosody & object size on lip opening +formant frequencies - but is f0/head angle 'direct' coupling?

The Role of Speech Reading During Visual Word Processing in Hearing Children pubs.asha.org/doi/10.1044/... Is the STS process active in speech reading related to word reading? STS active in visual word-rhyming but only weakly linked to reading -Seems STS phonological processing a better candidate

Video-call glitches trigger uncanniness and harm consequential life outcomes www.nature.com/articles/s41... 5 experiments + 3 supplementary ones using both live & recorded interactions suggest that minor AV glitches during video calls can harm interpersonal judgements in consequential life domains

Rhythm Processing Across Development: Origins, Links to Language Processing, and Perspectives for Intervention nyaspubs.onlinelibrary.wiley.com/doi/10.1111/... Review article discusses evidence of rhythm processing in adults + children & extends this to early infancy.

Dynamic functional connectivity following cochlear implantation: Predicting speech outcomes and exploring brain network dynamics www.pnas.org/doi/10.1073/... Nearly 30% of CI people get limited benefit - fNIRS study measured brain activity for AO & VO speech at 1 mo & 1 y postimplantation ...

Very happy to see @vtolkacheva.bsky.social's paper on the role of sublexical information in perception and misperception of speech now open access in Journal of Memory & Language: www.sciencedirect.com/science/arti... @profkatie.bsky.social (7 experiments!!)

🔬 Excited to present our latest research at the #MAIN2025 conference today! 🔗 laurentperrinet.github.io/talk/2025-12... #DeepLearning #ComputerVision #AI #Research #NeuralNetworks #NeuroAI #OpenScience I love #Montreal

Now published in the Journal of Neurophysiology: journals.physiology.org/doi/full/10.... Get in touch if you think this tool could help in your science! We will be developing improvements and extensions over the next year.

Visual load does not modulate neural processing of audiovisual speech integration www.sciencedirect.com/science/arti... Neural "binding" of acoustic & visual speech signals not modulated by visual load, suggests results support an automatic integration of acoustic & visual speech cues

Meaningful results for meaningful hypotheses www.researchgate.net/publication/... Tutorial provides an accessible, non-technical introduction to a technique - Bayesian inference - that is both conceptually easy to understand, computationally cheap & covers common experimental scenarios 👏

MEG-SCANS - A comprehensive magnetoencephalography speech dataset with Stories, Chirps and Noisy Sentences https://pubmed.ncbi.nlm.nih.gov/41366218/

Multisensory integration for active mechanosensation in Drosophila flight www.cell.com/current-biol... "Together, these results reveal novel behavioral gating of sensory information and expand our understanding of the efferent control of active sensing" both visual presence+content influence flight

Three distinct components of pragmatic language use www.pnas.org/doi/10.1073/... Identified three interpretable components of pragmatic language use: 1) Adherence to social conventions 2) Getting meaning from intonation 3) causal reasoning from world knowledge....

InCHORRRuS: Infant-Directed Communication Highlights and Organizes Repetition and Redundancy Through Rhythmic Structure nyaspubs.onlinelibrary.wiley.com/doi/10.1111/... Reviews literature on timing & rhythm, redundancy & repetition in ID signals - redundancy & repetition ... redundancy & repetition

Effects of Audiovisual Asynchrony on Speech Intelligibility in Typically Hearing Adults and Cochlear Implant Users pubs.asha.org/doi/10.1044/... Used intelligibility to examine the interaction of listener & stimulus characteristics for asynchronous AV speech in CI users & typically hearing controls

Sounds easy, looks nice: Crossmodal transfer of auditory processing fluency to visual object preference link.springer.com/article/10.3... Manipulated perceptual fluency in a simple auditory task which participants performed while viewing visual objects, which they rated for liking ...

Talking Hands, Shifting Tongues:How Using Co-speech Gestures and Second Language Relate to Emotional Autobiographical Memory Narration? journals.sagepub.com/doi/10.1177/... Investigated how hand gestures, experience, narrative details, & emotionality interact during autobiographical memory narration

Brain Representation of Numerosity across the Senses and Presentation Formats www.jneurosci.org/content/45/4... How does the brain represents numerical information (range 2–5) across different modalities (auditory, visual) & presentation formats (sequential, simultaneous; symbolic, nonsymbolic)?

New work from the lab! A full multimodal picture of caregivers behaviours supporting learning in the early years answering the question: what do caregivers say and do to support their children’s word learning?

“The School Cafeteria Problem”: Disrupted Visuospatial Attention During Multisensory Speech-in-Noise Perception in Children with ADHD www.biorxiv.org/content/10.1... Children with ADHD show reduced perceptual benefit from exposure to multisensory compared to unisensory speech stimuli

Preconfigured neuronal firing sequences in human brain organoids www.nature.com/articles/s41... "...results suggest that temporal sequences do not arise in an experience-dependent manner, but are rather constrained by a preconfigured architecture established during neurodevelopment"

Fun read (Q and As) from the workshop: Speech Production Models and Empirical Evidence from Typical and Pathological Speech hal.science/hal-05352262...

for all of you using the ALIGN library (to measure lexical, syntactic and semantic alignment in conversations), Nick Duran has put together a great refactoring: ALIGN 2.0 (github.com/nickduran/al...), now integrated with Spacy and Bert

The Role of Modality-specific Brain Regions in Statistical Learning: Insights from Intracranial Neural Entrainment direct.mit.edu/jocn/article... iEEG to trisyllabic words - tracking at (approx) syllable & word frequencies - SL assessed explicit & implicit measures - SL found but modality-specific

AV integration and acoustic change complex in adult cochlear implant users: an electrophysiological and behavioral investigation www.tandfonline.com/doi/full/10.... CI users show limitations in AV integration & prolonged ACC latencies (reflecting auditory discrimination not multisensory integration)

The ADVANCE toolkit: Automated descriptive video annotation in naturalistic child environments link.springer.com/article/10.3... Looks like: YOLOv8; MediaPipe (used to mask frames); OSNet; OpenPose (via Docker) & some k-means clustering & XGBoost - code @ osf.io/4mfsk/ Hmm - info on setting up?

Joint Population Coding and Temporal Coherence Link an Attended Talker’s Voice and Location Features in Naturalistic Multi-talker Scenes www.jneurosci.org/content/45/4... Two complementary mechanisms: joint population coding & temporal coherence enable integration of voice & location features

Head Gestures Do Not Serve as Precursors of Prosodic Focus Marking in the Second Language as They Do in the First Language onlinelibrary.wiley.com/doi/pdf/10.1... "L2 learners highlight the same, possibly inaccurate, part of an utterance in both modalities"

Audiovisual Speech Perception With Less Familiar and Frequent Words pubs.asha.org/doi/full/10.... Investigated how word familiarity affected word recognition in A-only & AV conditions using medically related sentences embedded in a simulated hospital soundscape

Front page note on Perceptrons draft "We are anxious to collect: ...3. statements that are clear but false....Diligence will be rewarded, somehow or other" : )

Neural Tracking of the Maternal Voice in the Infant Brain www.jneurosci.org/content/earl... Used TRF to look at how 7-month-old human infants track maternal vs unfamiliar speech & if this affects simultaneous face processing - maternal speech enhances neural tracking & alter how faces are processed

Visual Speech Reduces Cognitive Effort as Measured by EEG Theta Power and Pupil Dilation www.eneuro.org/content/12/1... Combined pupillometry & EEG to investigate how visual speech cues modulate cognitive effort during speech recognition ... code/software & data github.com/brianman515/... - nice!

happy to share our new paper, out now in Neuron! led by the incredible Yizhen Zhang, we explore how the brain segments continuous speech into word-forms and uses adaptive dynamics to code for relative time - www.sciencedirect.com/science/arti...

Correlation detection as a stimulus computable account for AV perception, causal inference & saliency maps in mammals elifesciences.org/articles/106... Image- & sound-computable population model for AV perception -> Used simulation to model psychophysical, eye-tracking & pharmacological experiments

A richly annotated dataset of co-speech hand gestures across diverse speaker contexts www.nature.com/articles/s41... Dataset comprising 2373 annotated gestures, 9 speakers across 3 distinct categories: University lecturers, Politicians, and Psychotherapists can be accessed at doi.org/10.17605/OSF...

Applications are now open for the MARCS International Visiting Scholar Program 2026! 🌏 We are pleased to offer scholarships for PhD students and postdocs for visits of 1–3 months before the end of 2026. 📅 Applications close 4 December 2025. If you are interested, email [email protected]

Distinct Portions of Superior Temporal Sulcus Combine Auditory Representations with Different Visual Streams www.jneurosci.org/content/45/4... Analysed open-source auditory cortex fMRI data from people watching a movie with ANNs to investigate STS & 2 visual streams

When brain talks back to the eye "The state of our brain shapes what we see, but how early in the visual system does this start? A new study in PLOS Biology shows that brain state-dependent release of histamine modulates the very first stage of vision in the retina" journals.plos.org/plosbiology/...

Here's an interesting new study exploring whether LLMs are able to understand the narrative sequencing of comics and... even the best AI models are *terrible* at it for pretty much all tasks that were analyzed aclanthology.org/2025.finding...

Can you feel what I am saying? Speech-based vibrotactile stimulation enhances the cortical tracking of attended speech in a multi-talker background https://www.biorxiv.org/content/10.1101/2025.10.31.685484v1

Audiovisual Synchrony in Left-hemisphere Brain-lesioned Individuals with Aphasia direct.mit.edu/jocn/article... Found a statistically significant effect of aphasia type on measures of AV synchrony; an effect not explained by lesion volume...damage to left posterior temporal bad for AV processing.

Expectation-driven shifts in perception and production pubs.aip.org/asa/jasa/art... Failed to find evidence that individuals' expectation-driven shifts in perception correlate with those in production ...

Yes, the McGurk effect, that's right -> "The influence of age, listener sex, and speaker sex on the McGurk effect" journals.sagepub.com/doi/10.1177/... Are reports of higher sensitivity to the McGurk effect in females than males influenced by the match of Listener-Speaker sex?

Audiovisual speech perception in Mandarin cochlear implant users across age and listening conditions www.sciencedirect.com/science/arti... "AV cues play a critical role in speech perception for Mandarin-speaking CI users, especially under acoustically challenging conditions"

Visual induction of spatial release from masking during speech perception in noise pubs.aip.org/asa/jel/arti... "There was no enhancement of auditory SRM through visual spatial separation" (shown before) - It did have a negative effect though i.e., to "disrupt existing auditory SRM" ...

Attentional engagement with target and distractor streams predicts speech comprehension in multitalker environments https://pubmed.ncbi.nlm.nih.gov/41136338/

Visible Pre-acoustic Lip Motion Aids Listeners’ Judgments of Speech Onset Times www.researchgate.net/profile/Pete... Visible lip motion facilitates detecting acoustic speech onset ....