Leveraging a human brain tissue bank with cognitive data across the lifespan, we identify a role for oligodendrocytes — myelin-producing cells in the brain — in age-related cognitive decline. With worsening cognitive decline, oligodendrocytes become dysfunctional and produce myelin of poor integrity, and experimental modeling of this myelin pathology impairs cognitive performance in aging.