Humans explore unfamiliar options more when deciding how to avoid losses than when seeking gains. The firing rate of neuronal cells in the brain’s amygdala predicts decisions to explore in both learning strategies, but neuronal ‘noise’ predicts exploration decisions only when avoiding losses. Together, these signals explain the increased exploration when risking losses.