A new approach sheds light on the biological features learned by protein language models, promising greater interpretability for unsupervised sequence learning.