The 15-minute AI stress test e... Note

The 15-minute AI stress test every designer can run

The Spaghetti Table Protocol challenge invites users to test AI's understanding of physical reality. Recent pilot study results for GPT-4o, Gemini 1.5 Pro, and Claude 3.5 Sonnet showed significant limitations in generating physically coherent images. Despite flawless photorealistic generation, the AIs struggled with basic physics, scoring poorly on structural coherence. Claude 3.5 Sonnet linguistically identified the impossibility but still generated a flawed image. GPT-4o and Gemini 1.5 Pro lacked even symbolic awareness of the problem. The protocol measures continuity, gravity and physics, and reversibility of thought. Human embodied cognition instantly detects physical impossibilities, an ability currently absent in AI. The challenge encourages broader testing by the UX/UI and HCI community. Participants are asked to prompt an AI to generate a specific table and then simulate its collapse five seconds later. Results are scored based on whether the AI flagged the impossibility and if the collapse sequence followed physical logic. The study also checks for unintended element contamination from prior prompts. This collaborative effort aims to build a public dataset to understand AI's limitations.
CdXz5zHNQW_uw70CEngeN.jpeg