The HydroGym computational environment teaches agents strategies for fluid-flow control that can be applied to scenarios they have not encountered before.