Foundation models are becoming increasingly popular across various biological domains, yet rigorous methods to comprehensively evaluate their full potential remain underdeveloped. This Perspective discusses current limitations and challenges in benchmarking these models and proposes guidelines for more robust and meaningful evaluation.