How the Brain Learns to Predict What to Fear Based on Prior Adverse Experiences

New study by Rutgers researchers offers potential new ways to treat anxiety and PTSD

Researchers in the laboratory of John P. McGann, Professor in the Department of Psychology (Behavioral and Systems Neuroscience) at Rutgers New Brunswick, recently published a study in Nature Communications that reveals how the brain decides which sensory experiences previously associated with an adverse experience should trigger a fear response (learned fear response). When the mechanisms underlying this decision-making process break down, the learned fear response can drive maladaptive behaviors even when experiencing situations that do not predict a threat. Individuals with anxiety and post-traumatic stress disorders often exhibit these maladaptive behaviors.

A single sensory stimulus, whether an odor, a sound, or an image, always activates many neurons throughout the brain, which often overlap for stimuli that are similar. When the brain has to decide which stimuli to avoid after a bad experience, it must determine how much overlap in neural response is enough to treat not only the original threatening stimulus as dangerous, but also different yet similar stimuli; i.e., to distinguish the original versus related sensory stimuli (“sensory discrimination”), the brain must create boundaries (“fear boundaries”) to limit the neurons that are activated.

To understand how smell-associated fear responses to an original and related stimuli are processed in the brain, researchers of the McGann lab first trained mice to associate a specific odor with a mild threat and then measured neural activity as the mice responded both to the conditioned odor and to a gradient of new, similar, and dissimilar smells. Natalia Efimova, a first‑year PhD student in the McGann lab, contributed to the study by performing the cannula infusions and subsequent behavioral testing. They measured the neural activity in the olfactory bulb, a brain region involved in the initial processing of the sense of smell.  By delivering precise pharmacological micro-infusions of neuromodulatory drugs (such as GABA_B antagonists and agonists) directly into the olfactory bulb, the team manipulated local inhibitory circuits and tested whether early sensory processing in the olfactory bulb directly controls fear boundaries and sensory discrimination.

How fear spreads to stimuli never paired with threat

The study showed that, for smells, sensory discrimination and establishment of fear boundaries occur in the early sensory processing part of the brain. In vivo imaging revealed that fear learning reduces presynaptic GABA_B receptor-mediated inhibition at the olfactory nerve terminals. By manipulating this inhibitory circuit in the olfactory bulbs of mice, the researchers could artificially change how readily the animals generalized their fear to new but related odors. Fear learning naturally changes this inhibitory circuit throughout a set of neurons activated by the threat-predictive odor and also in some neurons physically adjacent to them. As a result, when the animal encounters a new but similar smell that activates an overlapping or neighboring set of neurons, some of those neurons have already been modified by the experience of danger. Depending on the degree of overlap, this can cause the new odor to be interpreted as threatening and helps explain why the animal exhibits a fear response to a stimuli that was never directly paired with the threat.

Overlap and proximity both shape how far fear spreads

Both overlapping neural representations and the physical proximity of neurons contributed to how far the fear effect spread. Different odors activate distinct but partially overlapping patterns of neurons in the olfactory bulb. Fear generalized more strongly to odors whose activity patterns overlapped with the threat-predictive smell, offering a natural mechanism for why similar stimuli more readily evoke fear.

At the same time, the effects also spread according to physical proximity of neural activity. The strongest changes occurred near regions representing the conditioned odor and decreased with distance. This suggests that the physical layout of which neurons sit next to which—alongside how much their activity patterns overlap—helps determine how fear spreads across odors.

Early sensory processing sets the boundary between dangerous and safe

After learning to associate a specific odor with threat, mice normally generalized fear along a gradient: strongest fear toward the odor paired with shock, less fear toward a similar odor, and the least fear toward a dissimilar one. When the inhibitory circuit (GABA_B receptors) in the olfactory bulb was artificially blocked with an antagonist drug, this gradient collapsed and fear became broadly generalized to all odors equally. Conversely, infusing a GABA_B activator or agonist (baclofen) narrowed the range of odors the mice avoided without impairing their ability to tell the odors apart. After the infusion, mice could still distinguish the odors as accurately as controls, showing they were not confused about stimulus identity. Yet they no longer treated as many of those odors as potential threats.

This finding suggests that the brain begins determining where to draw the boundary between dangerous and safe stimuli much earlier in sensory processing than previously expected. Inhibitory processing in the olfactory bulb is not simply passive transmission of sensory information; it actively modulates how far the sensory stimuli-induced neuronal activation spreads following a fearful experience, extending the influence of the experience across different stimuli. The sensory processing region itself helps decide whether a new smell is similar enough to a previously threatening odor to trigger fear, or different enough to be treated as safe.

The results of this new study suggest that overly generalized fears may result from neuroplasticity not only in higher brain regions known for emotion and memory, but also from early changes in how the brain represents the stimulus itself. Consequently, sensory training or sensory exposure therapy coupled with targeted therapeutic interventions using drugs that modulate GABAergic neurotransmission might be clinically beneficial in treating fear generalization in anxiety and post-traumatic stress disorders.

References:

• Bakir, A.K., Gaikwad, A.A., Efimova, N. et al. Generalization of fear learning is shaped by inhibitory sensory processing in mice. Nature Communications 17, 4825 (2026). https://doi.org/10.1038/s41467-026-71356-5
• McGann Laboratory: The Neuroscience of Smell