Perception & Brain Dynamics Laboratory Research

Every day, our brains cycle through different states of awareness: from the rich conscious experiences during wakefulness to dreamless sleep to the bizarre experiences during dreaming sleep. The neural basis of conscious awareness remains one of the lasting mysteries in neuroscience.

In the Perception and Brain Dynamics Laboratory at NYU Langone, we aim to elucidate the brain mechanisms underlying perception in humans. Our research is motivated by questions such as the following:

How does the brain generate vivid perceptual experiences?
What are the differences between conscious perception and unconscious processing?
How does the perceptual system interact with mnemonic and cognitive systems?
What aberrant neural mechanisms underlie perceptual disturbances such as hallucinations?

Answering these questions not only addresses a lasting mystery about the neurobiological origins of our conscious perceptual experiences but also has far-reaching impacts on understanding the roots of perceptual disturbances in neurological, psychiatric, and aging-related disorders.

Below are some highlights of our recent work.

Perceptual Awareness

In a recent 7T fMRI study, we showed that successful conscious recognition is a result of a tight coordination between multiple large-scale brain systems and subcortical brain structures (Levinson et al., Nat. Commun., 2021). Interestingly, cortical activity in multiple large-scale brain systems—including those activated and deactivated by the task—contains information about the content of perception, but subcortical activity does not contain such information. This work suggests that content-rich cortical activity and non-content-specific subcortical activity coordinate to generate conscious recognition.

We have also used EEG and MEG to probe the rich temporal dynamics of neural activity underlying conscious perception. In an earlier study, we showed that aperiodic neural activity in the low-frequency (<5 Hz) frequency range (the slow cortical potentials, SCP) encodes the content of conscious perception (Li et al., J. Neurosci., 2014). Recently, using the bistable perception paradigm, we replicated this finding and also demonstrated that the amplitude fluctuations of alpha (~10 Hz) and beta (~20 Hz) brain oscillations support the temporal stability of conscious perception (Zhu et al., Sci. Rep., 2022; Hardstone et al., eLife, 2022).

This picture shows a set of cortical (left panel) and subcortical (right panel) brain regions whose activity magnitudes differ between consciously recognized and non-recognized conditions, for the same set of object images. Regions in warm (/cool) colors have higher (/lower) activity in recognized trials compared to unrecognized trials. These results were obtained using the 7 Tesla MRI scanner at NYU Langone and published in Levinson et al., Nat. Commun., 2021.

Spontaneous Brain Activity’s Influence on Perception

Sensory input does not enter a silent brain; instead, the brain is always active. In a 2017 paper (Baria et al., PLoS Comput. Biol., 2017), we showed that large-scale activity pattern in the SCP range predicts the subjective perceptual outcome (“seen” vs. “unseen”) of an upcoming threshold-level visual stimulus up to 2 sec before stimulus onset. Depending on the initial brain state, an identical stimulus input can trigger large-scale brain activity to follow distinct trajectories in the state space, corresponding seeing or not seeing the stimulus. These observations fit with the framework of robust, transient dynamics (He, TiCS, 2018).

Using a threshold-level object recognition paradigm, we found that there are different types of spontaneous brain activity that influence perception in distinct manners: one type of spontaneous activity was linked to fluctuations of the arousal system and shifted participants’ perceptual detection criterion; another type of spontaneous activity was unrelated to arousal and impacted participants’ perceptual sensitivity (Podvalny et al., Nat. Commun., 2019). In a follow-up study, we systematically mapped out the relationship between arousal fluctuations and large-scale cortical activity changes (Podvalny et al., eLife, 2021). And a recent study using 7T fMRI mapped out the anatomical sources of these perceptually relevant spontaneous activity processes, which involve multiple brain networks (Wu et al., Nat. Commun., 2024).

This picture was published in a review article (McCormick, Nestvogel, He, 2020) and summarizes two studies from our lab showing the influences of pre-stimulus spontaneous activity on conscious visual perception. Left: Work from Podvalny et al., Nat. Commun., 2021, showing two types of spontaneous activity influencing conscious recognition in different manners. Right: Work from Baria et al., PLoS Comput. Biol., 2017, showing the influence of pre-stimulus brain activity on stimulus-evoked brain activity underlying conscious perception.

How Memories Influence Perception

Memories from past experiences powerfully shape our daily perception. Our recent work has elucidated the brain mechanisms subserving this phenomenon through a powerful laboratory paradigm that induces one-shot perceptual learning, whereby a single viewing experience drastically alters perception of related images thereafter. Using 7T fMRI, we showed that prior knowledge from one-shot learning alters neural activity underlying perception across widespread brain networks (González-García et al., eLife, 2018). The availability of relevant memories sharpens neural activity across this entire cortical hierarchy (González-García and He, J Neurosci., 2021). Using MEG, we found that feedback activity from higher-order brain regions guides processing in lower-order visual regions over a time window of 300–500 ms (Flounders et al., eLife, 2019). Finally, collaborating with Larry Squire’s lab at UCSD, we showed that amnesiac patients with bilateral hippocampal lesions nevertheless had completely intact one-shot perceptual learning (Squire et al., PNAS, 2021), demonstrating that one-shot perceptual learning is supported by the neocortex.

In parallel work, we investigated how memories from life-long experiences shape perception using intracranial recordings in neurosurgical patients undergoing invasive monitoring for clinical treatment. we found that, contrary to prevailing views emphasizing lower-order sensory areas, prior knowledge from lifelong experiences is carried by top-down feedback (Hardstone et al., Nat. Commun., 2021).

This is the famous “Dalmatian Dog” picture created by Richard Gregory around 1970. The picture contains a hidden dog, which, once recognized will remain easily recognizable.

Funding Sources

Our work is generously supported by the National Eye Institute (NIH), National Science Foundation, Templeton World Charity Foundation, W.M. Keck Foundation, Irma T. Hirschl Trust, and NYU Langone’s Parekh Center for Interdisciplinary Neurology. Our research has also benefited from funding by Feldstein Medical Foundation, Leon Levy Foundation, Klingenstein Fund, Simons Foundation, and FACES foundation.

Trainees in our lab have received funding from the global Marie Curie Fellowship (EU), Leon Levy Neuroscience Fellowship, Charles Revson Senior Fellowship in Biomedical Sciences, Astellas Foundation, and Uehara Memorial Foundation grants, as well as multiple NYU Dean’s Undergraduate Research Fund grants.

As of June 28, 2024.

Methodologies

Our lab uses multimodal human brain imaging, including 3T and 7T fMRI, non-invasive and invasive electrophysiology (e.g., EEG, MEG, ECoG), combined with brain stimulation (e.g., tDCS, TMS) and computational modeling to study the neural bases of human perception.