The comparative performance and generalisability of pathology foundation models remain largely unexamined. Here, the authors benchmark 32 AI pathology foundation models across large cancer datasets, showing that generalisation in computational pathology is heterogeneous and task-dependent regardless of dataset scale, but ensemble-based approaches can combine the strengths of different models.