FDA's New Drug Approval AI Is Generating Fake Studies

Robert F. Kennedy Jr. advocated for AI tools like Elsa within agencies like the FDA. CNN reports that the FDA's AI tool, Elsa, is fabricating nonexistent studies and misrepresenting research. FDA employees have noted Elsa's tendency to "hallucinate" information and provide unreliable outputs. This raises concerns about the accuracy of information generated by the AI, especially when double-checking is limited. Kennedy's MAHA commission previously released a report citing fake studies, raising further suspicion. The FDA initially touted Elsa as cost-effective, but accuracy issues are overshadowing these benefits. Concerns exist about the potential consequences of using AI-generated, inaccurate studies for public health decisions. Checking AI-generated summaries against original reports is essential, but time-consuming. Elsa's errors, even when corrected, highlight the limitations of relying solely on AI for critical tasks. The reliance on flawed AI tools could compromise the FDA's regulatory processes and public trust.